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The  Pyramids,  the  Colossus  of  Rhodes,  the 
Statute  of  Zeus  at  Olympia,  and  Modeling  and 
Simulation  (M&S)  are  all  considered  wonders 
of  the  world.  Well,  maybe  M&S  is  not  actually 
a  wonder  of  the  world,  but  it  should  be.  Everyone  con¬ 
stantly  claims  how  wonderful  it  is  and  yet  no  one  can  real¬ 
ly  explain  what  it  is  or  how  it  works. 

Modeling  and  simulation  is  probably  one  of  today’s  most 
misunderstood  tools  that  we  use  in  aircraft  design.  Its  util¬ 
ity  is  rated  somewhere  between  the  ultimate  tools  capable 
of  easily  designing  an  entire  aircraft  and  support  systems, 
to  a  necessary  evil  that  must  be  included  in  the  design 
process  to  satisfy  the  technology  junkies  of  the  21st 
Century.  Some  model  managers  claim  that  their  code  can 
leap  tall  building  in  a  single  bound  or  lasso  the  sun  and 
control  the  universe.  While  others  think  that  money  spent 
on  M&S  development  would  be  better  spent  on  building 
a  bridge  to  the  moon.  The  truth  is,  M&S  capabilities  are 
somewhere  near  the  middle.  Modeling  and  simulation  can 
be  a  great  tool,  but  a  model’s  limitations  and  assets  need 
to  be  well  understood  for  it  to  help  rather  than  hinder  the 
design  process.  Furthermore,  a  designer  must  remember 
that  M&S  is  a  tool  that  provides  input  into  the  organical¬ 
ly  based  (human)  decision  process.  Software  cannot  make 
decisions  for  us,  but  models  can  provide  very  good  inputs 
into  the  decision  making  process. 

The  Joint  Technical  Coordinating  Group  on  Aircraft 
Survivability  (JTCG/AS)  sponsors  the  development  and 
maintenance  of  a  number  of  aircraft  survivability  models 
including  ESAMS,  AJEM,  RADGUNS,  and  ALARM.  We 
have  worked  hard  over  the  years  to  make  sure  that  each 
model  is  the  most  capable  and  beneficial  to  the  analyst  by 
incorporating  the  most  recent  updates  and  maintaining 
the  latest  version  in  the  Survivability/Vulnerability 
Information  Analysis  Center  (SURVIAC)  for  distribution 
to  anyone.  Although  we  have  come  a  long  way  in  devel¬ 
oping  capable  models,  there  are  still  many  issues  that  exist 
for  the  M&S  community. 

This  newsletter  issue  focuses  on  many  of  the  great  uses  of 
M&S  in  aircraft  survivability  design  and  the  issues  sur¬ 
rounding  them.  One  of  the  articles,  “The  M&S 
Credibility  Workshop  II,”  was  written  by  our 
Survivability  Assessment  Subgroup  Chairman,  Mr.  Ron 
Ketcham.  He  describes  one  of  the  biggest  issues  in 
M&S — model  credibility.  Making  a  model  is  easy,  but 
making  one  that  is  accurate,  efficient,  and  reliable  is  a 
whole  other  story.  The  JTCG/AS  has  spent  a  lot  of  time 
and  money  on  refining  many  of  the  models  used  in  air¬ 
craft  design  so  that  aircraft  designers  and  program  offices 
are  comfortable  using  these  models  as  a  part  of  their  deci¬ 
sion  processes.  If  we  can  make  models  credible  and 
reusable,  we  can  save  a  tremendous  amount  of  precious 
resources,  both  financial  and  human,  that  can  be  applied 
to  other  issues  in  the  design  field. 

Another  issue  facing  the  M&S  community  is  how  to  transition 
from  legacy  code  to  next  generation  models.  Just  look  at  all  the 


heartburn  JMASS  is  causing.  It  is  obvious 
that  we  have  the  need  and  the  technology  to 
produce  more  capable,  cost  efficient  codes  that 
can  replace  multiple,  older,  and  less  capable  codes. 

But  how  to  do  this  is  the  problem.  I  liken  the  tran¬ 
sition  from  legacy  codes  to  next  generation  models 
to  a  young  Tarzan  swinging  through  the  jungle.  At 
some  point,  Tarzan  grabs  hold  of  the  next  vine  that 
doesn’t  look  steady,  so  he  clings  to  the  vine  behind  him. 
He  needs  to  continue  going  forward  to  make  progress 
through  the  jungle  but  hanging  on  to  these  two  vines 
paralyzes  him.  What  should  Tarzan  do?  During  a  brief¬ 
ing  to  senior  DOT&E  leadership,  I  asked  this  same  ques¬ 
tion.  The  answer  was  easy.  “Just  cut  the  old  vine.”  That 
wasn’t  the  answer  I  expected  but  it  made  the  point.  We 
need  to  find  a  more  efficient  way  to  speed  up  the  transi¬ 
tion  from  legacy  code  to  newer  more  capable  models 
while  not  losing  any  capabilities.  Money  is  a  finite 
resource,  and  we  cannot  afford  to  support  and  upgrade 
existing  models  while  developing  more  advanced  ones. 
There  is  not  enough  money  to  go  around.  Add  in  spi¬ 
raling  costs  and  production  delays  in  most  software 
acquisitions,  and  it  is  understandable  why  senior 
decision-makers  become  impatient  with  current 
efforts  at  model  development.  One  of  the  articles  in 
this  newsletter  focuses  on  the  Advanced  Joint 
Effectiveness  Model  (AJEM)  development,  which  is 
a  joint  effort  between  the  JTCG/AS,  JTCG/ME, 
and  the  ARL.  Although  we  have  made  some  good 
progress  in  the  AJEM  development,  the  JTCG/AS 
has  learned  firsthand  all  the  headaches  that 
accompany  any  model  transition. 

When  I  first  got  to  this  job,  I  received  the 
M&S  tutorial — fire  hose  style.  I  was  secretly 
taken  to  a  dark  basement  office  in  some 
building  at  Wright-Patterson  AFB,  where 
toothpicks  were  used  to  keep  my  eyes  open, 
and  I  was  briefed  for  hours  on  end  about  all 
the  different  models  and  what  they  were  used 
for.  Now  I  have  an  appreciation  for  how 
M&S  works  and  why  we  need  so  many  dif¬ 
ferent  models.  I  am  nowhere  near  being  an 
M&S  expert  but  I  now  know  enough  to  under¬ 
stand  that  M&S  has  its  place  in  this  world.  The 
trick  is  using  models  correctly  by  providing  the 
appropriate  documentation  and  code  validation 
and  verification  to  aid,  not  inhibit,  the  decision¬ 
making  process.  To  this  end,  the  JTCG/AS  is  com¬ 
mitted  to  insure  that  the  decision-makers  have  the 
best  tools  available. 


/%, 


LCDR  Andrew  (Andy)  Cibula  (USN) 
Director,  JTCG/AS  Central  Office 
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The  M&S  Credibility  Workshop  II 


Planning  for  the  Credible  Employment  of  M&S  in  Acquisition 


■  by  M r.  Ron  Ketcham 


The  second  workshop  for 
the  promotion  of  modeling 
and  simulation  (M&S) 
credibility  was  held  at  the 
Silver  Legacy  Hotel  in  Reno,  Nevada 
on  5-7  March  20002.  This  work¬ 
shop,  titled  “Planning  for  the 
Credible  Employment  of  M&S  in 
Acquisition,”  was  sponsored  by  the 
Joint  Technical  Coordinating  Group 
for  Aircraft  Survivability  (JTCG/AS). 
The  M&S  Credibility  Workshop  was 
facilitated  by  the  Joint  Accreditation 
Support  Activity  (JASA),  and 
endorsed  by  the  Defense  Modeling 
and  Simulation  Office  (DMSO)  and 
the  National  Defense  Industry 
Association  (NDIA).  Approximately 
125  persons,  representing  a  broad 
spectrum  of  Government  and  indus¬ 
try,  attended  the  workshop. 

Planning  for  the  Workshop: 
The  WEAC 

The  second  workshop  (WS  II)  on 
M&S  credibility,  was  a  true  out¬ 
growth  of  the  first  workshop  held  in 
February  2001.  During  the  first 
workshop  (WS  I),  many  of  the  prob¬ 
lems  and  issues  plaguing  M&S  credi¬ 
bility  were  addressed  and  document¬ 
ed.  The  goal  for  WS  II  was  to  con¬ 
centrate  on  identifying  solutions  and 
then  follow  up  the  workshop  with  a 
plan,  or  roadmap,  to  start  the  long 
process  of  seeking  to  implement 
these  solutions. 


After  the  initial  workshop  (Aircraft 
Survivability,  Fall  2001)  JASA  decided 
to  expand  participation  in  the  plan¬ 
ning  process  by  creating  a  steering 
committee  with  membership  from 
throughout  the  Department  of 
Defense  (DoD).  The  goal  was  to 
broaden  the  perspective  of  the  work¬ 
shop  and  enhance  the  probability  of 
success  by  reaching  out  to  the  various 
M&S  stakeholders.  This  committee 


Rear  Admiral  Robert  E.  Besal  delivering  workshop  keynote  on  “An  Appropriate  Role  for  M&S 
in  Operational  Test:  One  Tester’s  Perspective.” 


was  created  in  late  summer  of  2001 
and  was  named  the  Workshop 
Executive  Advisory  Committee 
(WEAC,  pronounced  Wee-Ack).  The 
membership  of  the  WEAC  included — 

•  Ron  Ketcham,  Chairman  (JASA) 

•  Bob  Cook  (SAF/AQI) 

•  Doug  Fraedrich  (NRL) 

•  Frank  Gray  (AFOTEC) 

•  Brian  Hall  (OPTEVFOR) 

•  John  Haug  (ATEC) 

•  Hans  Mair  (IDA — in  support  of 
OSD) 

•  Debra  Ridgeway  (AMSO) 

•  Jim  Sebolka  (Barcroft  Group) 


•  Jack  Sheehan  (DOT&E,  C3I  & 
Strategic  Systems) 

•  Tracy  Sheppard  (University  of 
Texas,  Austin — in  support  of 
OSD) 

•  Ron  Thompson  (JTCG/ME 
Central  Office) 

•  Mike  Weisenbach  (JTCG/AS) 

•  Simone  Youngblood  (DMSO) 

•  Kathy  Russell,  Workshop 
Coordinator  (JASA) 

The  WEAC  met  several  times 
between  the  two  workshops  to  define 
the  objectives  and  processes  of  WS  II, 
drafted  and  published  a  call  for 
papers,  selected  and  invited  special 
presenters  and  the  keynote  speaker, 
and  set  the  final  agenda.  WEAC 


members  also  served  as  chairmen  of 
five  sessions  of  WS  II.  Special  men¬ 
tion  goes  to  the  participation  of 
DMSO  in  the  planning  and  prepara¬ 
tion  of  WS  II.  Ms.  Simone 
Youngblood,  the  DMSO 
Verification,  Validation  and 
Accreditation  (VV&A)  Technical 
Director,  hosted  and  facilitated 
WEAC  meetings  at  DMSO  head¬ 
quarters.  She  has  also  assumed  an 
even  greater  role  in  the  post  work¬ 
shop  efforts  by  taking  the  lead  in 
documenting  what  was  learned  at 
WS  II  and  developing  a  roadmap  to 
promote  more  credible  employment 
of  M&S. 

The  Workshop 

Although  WS  II  did  not  officially 
begin  until  Tuesday,  5  March  2002, 
early  arrivals  were  able  to  attend  two 
tutorials  held  on  Monday.  Over  half 
of  the  WS  II  attendees  took  advan¬ 
tage  of  these  events.  The  initial  tuto¬ 
rial  was  “The  Statistics  of 
Validation”  presented  by  Dr.  Alan 
Breitler,  recently  retired  from  the 
Center  for  Naval  Analysis  (CNA). 
Michelle  Kilikauskas,  JASA  Director, 
also  presented  a  tutorial  on  W&A. 

The  workshop  sponsors  had  a  more 
visible  presence  and  impact  at  WS  II. 
The  first  day  began  with  an  opening 
presentation  by  the  JTCG/AS  Central 
Office.  LCDR  Andy  Cibula,  Director 
of  the  JTCG/AS  Central  Office,  began 
by  detailing  the  function  and  struc¬ 
ture  of  the  JTCG/AS.  He  described 
the  historical  role  that  the  JTCG/AS 
has  had  with  M&S.  The  JTCG/AS 
currently  has  a  major  role  in  model 
development,  with  such  efforts  as 
AJEM  (the  Advanced  Joint  Endgame 
Model).  It  also  plays  a  pivotal  role  in 
M&S  support  through  Survivabil¬ 
ity/Vulnerability  Information  Analy¬ 
sis  Center  (SURVIAC),  which  in  addi¬ 
tion  to  being  a  repository  of  M&S 
used  in  survivability,  also  sponsors 
user  group  meetings  to  promote  com¬ 
munity  involvement  in  the  manage¬ 
ment  and  direction  of  M&S  tools  by 
the  appropriate  subject  matter 
experts  (SMEs).  The  JTCG/AS  has 
also  promoted  M&S  credibility  with 
the  creation  of  JASA  and  other  efforts 
such  as  the  ESAMS  Cooperative 
Assessment  Team  (ECAT).  ESAMS  is 
a  group  of  subject  matter  experts 
using  open-air  test  range  data  to  com- 


From  Left  to  Right:  Col  Phil  Faye  (ESC/CX),  LCDR  Andy  Cibula  (JTCG/AS),  Ron  Thompson 
(JTCG/ME),  Col  Kevin  Greaney  (MDA),  Chad  Holvenstot  (SAIC),  Edmond  Levy  (Northrop 
Grumman),  David  Blancett  (Northrop  Grumman),  Maj  Steven  Bishop  (USF  PO). 


pare  with  ESAMS  outputs  for  valida¬ 
tion  purposes.  But  there  are  concerns 
within  DOT&E  and  the  JTCG/AS 
regarding  the  rising  costs  of  M&S 
support.  The  JTCG/AS  needs  to 
reevaluate  the  role  they  have  in  M&S 
management  and  support.  LCDR 
Cibula  asked  the  conference  atten¬ 
dees  “who  is  in  charge  of  M&S  with¬ 
in  the  DoD?”  He  asked,  “what  was 
the  role  of  DMSO  and  who  should  be 
sponsoring  this  Workshop?”  He  also 
said  that,  “there  needs  to  be  a  plan 
for  the  long-range  support  of  M&S.” 
Finally,  he  asked,  “what  was  going  to 
be  the  product  of  the  workshop?” 


(LFT&E)  office.  He  began  by  stress¬ 
ing  that  the  workshop  needs  to  have 
a  product.  The  product  needs  to 
focus  beyond  the  problems,  beyond 
the  possible  solutions,  and  place  aim 
at  actual  fixes.  He  also  raised  the 
issue  of  who  in  DoD  should  be  spon¬ 
soring  this  workshop. 

Wojo  also  identified  many  of  the 
T&E  community  concerns  with 
M&S— 

•  M&S  often  does  not  provide  the 
information  that  helps  to  answer 
the  key  programmatic  issues. 


At  his  luncheon  address  later  that 
day,  former  LFT&E  Director  James 
O’Bryon  answered  some  of  the  issues 
raised  about  the  role  of  DMSO.  He 
stated,  “there  are  differing  percep¬ 
tions  of  DMSO’s  roles.”  He  added 
that  the  role  and  size  of  the  DMSO 
staff  is  not  clearly  understood 
throughout  DoD.  DMSO  does  not 
have  the  people  to  take  on  M&S  in 
the  manner  desired  by  many — but 
not  all.  Finally,  he  challenged  the 
audience  with  the  statement  that  you 
should  “get  busy  solving  your  own 
problems,  do  not  look  for  OSD  or 
DMSO  to  solve  your  problems.” 

These  issues  were  further  addressed 
on  the  second  day  of  WS  II  by  Robert 
“Wojo”  Wojciechowski,  DOT&E 
staff  assistant  to  the  Live  Fire  Test 


•  There  are  few,  if  any,  early  V&V 
opportunities. 


•  There  are  no  tests  conducted  to 
generate  data  needed  for  model 
inputs. 

•  M&S  is  used  too  late. 

•  Model  predictions  are  often 
made  using  different  M&S  tools 
and  are  not  comparable. 

•  There  is  a  lack  of  a  model-test- 
model  approach. 

•  Poor  model  documentation  often 
does  not  provide  the  information 
required  to  assist  the  user  in 
determining  M&S  credibility. 
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•  There  are  difficulties  and  limita¬ 
tions  in  correlating  M&S  results 
with  test  results. 

•  Proprietary  M&S  often  prevents 
a  credible  V&V  effort. 

•  M&S  tools  are  modified  for  each 
program,  thereby  making  past 
V&V  information  less  useful. 

•  “Building  block”  approach  is 
used  for  testing  but  not  M&S 
applications. 

He  ended  his  brief  with  some  clear 
recommendations  for  improving 
M&S  employment  in  test  and  evalu¬ 
ation  (T&E).  The  M&S  community 
should  be  included  in  the  TEMP 
process  to  ensure  that  M&S  is  used 
appropriately.  Improve  the  quality  of 
the  M&S  documentation  to  include 
assumptions  and  limitations,  and 
clearly  define  the  intended  applica¬ 
tions  for  the  model  or  simulation. 
Finally,  ensure  that  the  program  has 
supplied  resources  for  M&S  early 
enough  to  enable  testing  to  provide 
M&S  inputs  and  data  for  V&V,  to 
provide  for  the  development  of  any 
new  features  required  by  the  pro¬ 
gram,  and  finally  to  fund  M&S 
W&A  efforts. 

The  keynote  speaker,  Rear  Admiral 
Robert  E.  Besal  (COMOPTEVFOR) 
also  addressed  the  question  of  the 
role  of  M&S  in  T&E  during  his 
remarks  (the  memorandum  was 
signed  by  the  four  heads  of  the  OTA 
organizations  expressing  their  con¬ 
cerns  about  the  appropriate  role  of 
M&S  in  OT&E.)  They  were  con¬ 
cerned  that  with  the  pressure  to 
reduce  acquisition  time  and  costs, 
that  model  results  may  sometimes  be 
substituted  for  test  results.  RADM 
Besal  and  the  other  signatories  felt 
that  while  there  was  an  appropriate 
role  for  M&S  in  OT&E,  it  is  essen¬ 
tial  that — 

operational  testing  should  continue  to 
be  the  primary  basis  for  OT&E  [in 
order  to]  ensure  that  the  real  system, 
in  the  real  environment,  with  the  real 
operator,  can  accomplish  its  opera¬ 
tional  task. 

He  also  expressed  concern  that  the 
resources  were  not  being  made  avail¬ 


able  to  address  credibility  concerns 
in  a  timely  manner  for  OT&E. 

The  rest  of  the  briefs  were  divided 
into  the  five  Workshop  sessions. 
These  sessions  were — 

•  Programmatic  Issues  that 
Impact  Effective  Planning  for 
Credible  M&S  (Chairperson: 
Mr.  Mike  Weisenbach, 
JTCG/AS) —  This  session  exam¬ 
ined  current  acquisition  prac¬ 
tices  and  how  they  impact  the 
ability  to  plan  for  the  employ¬ 
ment  of  credible  M&S  in  surviv¬ 
ability  and  lethality. 

•  Effective  Planning  that  Promotes 
Credible  Utilization  of  M&S  in 
T&E  (Chairperson:  Mr.  Tracy 
Sheppard  University  of  Texas, 
Austin) — During  WS  I,  the  T&E 
community  specified  a  solid 
requirement  that  M&S 
employed  in  support  of  T&E 
must  be  credible.  This  session 
focused  on  how  the  T&E  com¬ 
munity  effectively  plans  for  the 
usage  of  M&S  and  how  their 
credibility  concerns  were,  or 
were  not,  being  addressed. 

•  Software  Development  and 

Management  Processes  that 
Promote  M&S  Credibility, 
Chairperson:  Mr.  Douglas 

Fraedrich — Naval  Research 
Laboratory  (NRL).  This  session 
examined  the  available  infra¬ 
structure  for  development  and 
maintenance  of  M&S  is  often 
extremely  limited.  This  signifi¬ 
cantly  impacts  the  ability  to 
properly  W&A  M&S  utilized 
in  survivability  and  lethality. 

•  V&V — Addressing  the  Technical 
Issues  (Validity,  Interoperability, 
and  Reuse)  (Chairperson:  Ms. 
Simone  Youngblood,  DMSO) — 
This  session  looked  at  technical 
issues  and  innovation  that  had 
impact  on  M&S  credibility. 

•  Planning  for  the  Transition  of 
Legacy  Models  to  New  Model 
Developments  (Chairperson: 
Ms.  Simone  Youngblood, 
DMSO) — The  survivability  and 
lethality  communities  are  facing 
a  major  M&S  tool  transition 


with  the  employment  of  the  new 
models  (e.g.,  AJEM,  JMASS, 
JSIMS,  JWARS)  to  replace  some 
current  legacy  models.  This  ses¬ 
sion  allowed  the  user  communi¬ 
ty  to  address  credibility  concerns 
with  regard  to  this  transition. 

Post  Workshop  Efforts 

The  critical  post  WS  II  follow-up 
activities  of  the  WEAC  began  at  the 
workshop.  The  WEAC,  joined  by 
other  interested  attendees,  met  at  the 
end  of  each  day’s  briefings  to  create  a 
daily  summary  of  the  key  points 
raised  during  the  daily  briefs,  panels, 
and  open  discussions.  This  was  a  key 
resource  in  developing  the  workshop 
out  brief. 

Special  mention  goes  to  Mr.  Jim 
Hollenbach,  President,  Simulation 
Strategies,  Inc.  Mr.  Hollenbach,  a  for¬ 
mer  Navy  Captain  and  Director  of 
DMSO,  formed  a  working  group  of 
WS  II  attendees,  exclusive  of  the 
WEAC  membership,  to  meet  after 
hours  to  address  some  of  the  issues 
being  raised  at  WS  II.  His  working 
group  reported  their  findings  on  the 
final  day.  One  of  the  key  concerns 
raised  by  this  working  group  was  that 
the  focus  of  the  WS  II  might  be  too 
broad  to  be  effective.  They  felt  a  bet¬ 
ter  approach  would  be  to  form  small¬ 
er  workshops  with  a  narrower  focus. 
Speakers  and  attendees  would  also  be 
more  selective  to  address  the  specific 
issue  in  a  more  substantive  way. 

Mr.  Hollenbach  has  joined  the 
WEAC  in  developing  the  WS  II  out 
brief  and  roadmap.  This  effort,  led 
by  Simone  Youngblood  of  DMSO,  is 
intended  to  analyze  and  digest  the 
briefs,  panels,  open  discussions,  and 
summary  slides  generated  at  WS  II  to 
identify  commonality  in  issues  and 
observations.  It  also  will  focus  on 
possible  resolution  identified  at  WS 
II  and  in  the  post  workshop  efforts. 
This  out  brief  will  form  the  basis  for 
the  roadmap  on  what  and  how  the 
WEAC  will  seek  to  address  these 
concerns.  ♦ 


After  Hours:  The  Workshop  generated  enthusiasm  that  lasted  well  after  the  daily  briefs.  This 
photo  was  taken  at  about  6:00  p.m.  after  the  second  session  on  Wednesday  6  March, 

2002.  It  depicts  multiple  workgroups  discussing  workshop  issues  after  hours. 


Mr.  Ron  Ketcham  has  a  B.A.  degree  from 
Transylvania  University  ( Lexington , 
Kentucky)  in  Economics,  and  a  MS.  in 
Economics  and  Industrial  Engineering  from 
Iowa  State  University.  He  began  his  career 
at  the  Naval  Weapons  Center  at  China 
Lake  in  1982  as  a  warfare  analyst  (Air-to- 
Air  and  Strike  Warfare)  for  the  Weapons 
Planning  Group.  In  1981 ,  he  took  over  as 
Project  M  anager  for  the  Analysts' 
WorkBench  (AW B).  During  this  period  he 
did  work  in  the  area  of  Software 
Development,  Software  Development 
Process  Improvements,  and  W&A.  In  the 
early,  1990s  Ron  served  on  a  DON  com¬ 
mittee  that  drafted  initial  Navy  M&S  W&A 
policies.  In  1997,  Ron  joined  JASA  and  took 
over  as  Accreditation  Lead  for  the  Joint 


Strike  Fighter  (JSF).  He  is  currently  head  of 
the  System  Survivability  Integration  Branch 
at  the  Naval  Air  Warfare  Center  Weapons 
Division  (NAWCWD  Code  418100D).  He 
is  also  the  Chairman  of  the  Survivability 
Assessment  Subgroup  for  the  Joint 
Technical  Coordinating  Group  on  Aircraft 
Survivability  (JTCG/AS). 


CDs  of  all  workshop  presentations,  out 
brief,  and  other  WS  II  related  documents 
can  be  obtained  by  contacting- 

Kathy  Russell 

Phone:  760.939.4908 

DSN:  437.4908 

E-mail:  RussellKA2@  navair.navy.mil 

Additional  information  on  postWS  II  efforts 
can  be  obtained  from— 

Ron  Ketcham 

Joint  Accreditation  Support  Activity 
Head,  Survivability  Integration  Branch 
NAWCWD  China  Lake,  C A 
Phone:  760.939.2363 

DSN:  437.2363 

E-mail:  ketchamrl@  navair.navy.mil 


SimoneYoungblood 

Defense  Modeling  and  Simulation  Office 
(DM  SO) 

Phone:  703.824.3436 
E-mail:  syoungblood@  dmso.mil 


LCDR  Andy  Cibula,  Director  of  the  JTCG/AS 
Central  Office 
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Modeling  and  Simulation  Credibility 

| 

■  by  M r.  Ron  Ketcham  and  Ms.  Simone  Youngblood 


The  Modeling  and  Simu¬ 
lation  (M&S)  Credibility 
Workshop  II  (WS  II), 
“Planning  for  Credible 
Employment  of  M&S  in 
Acquisition”  (March  2002)  provided 
attendees  from  Government  and 
industry  with  an  excellent  opportuni¬ 
ty  to  exchange  ideas  and  lessons 
learned  for  enhancing  the  quality  of 
M&S  results  throughout  the 
Department  of  Defense  (DoD).  WS  II 
also  provided  attendees  a  venue  for 
building  networks  of  M&S  profes¬ 
sionals  in  the  survivability,  lethality, 
and  mission  effectiveness  disciplines. 
However,  the  benefits  of  WS  II  were 
not  limited  to  the  actual  event.  It  was 
always  envisioned  that  WS  II  would 
begin  a  process  to  identify  obstacles 


(and  solutions)  to  the  credible  appli¬ 
cation  of  M&S,  and  the  ultimate 
goal  of  the  workshop  was  to  define  a 
roadmap  to  implement  process  and 
practice  improvements  that  mitigate, 
or  even  eliminate,  these  obstacles. 

In  order  to  accomplish  that  goal,  the 
Workshop  Executive  Advisory 
Committee  (WE AC)  immediately  fol¬ 
lowing  WSII  began  sifting  through  the 
workshop  presentations,  panel  discus¬ 
sion  notes,  and  attendee  comments  to 
pull  out  common  observations,  and 
themes,  and  document  recommended 
solutions  to  .  Over  a  period  of  several 
meetings  these  were  discussed  and  pri¬ 
oritized  by  members  of  the  WEAC. 
The  roadmap  is  being  created  from 
these  prioritized  lists  to  define  how  to 


implement  the  recommendations  from 
WS  II.  The  WEAC  also  took  into 
account  the  lessons  learned  from  the 
initial  M&S  Credibility  Workshop 
held  in  2001  and  the  follow-up 
actions  that  resulted  from  that  event 
to  further  enhance  the  probability  of 
success.  The  results  of  WS  II  was  pre¬ 
sented  to  the  Defense  Modeling  and 
Simulation  Office  (DMSO)  W&A 
Technical  Working  Group  (TWG)  in 
August  2002.  An  overview  of  the 
findings  to  date  is  presented  in  this 
article.  The  results  and  the  roadmap 
were  not  yet  complete  as  of  this  writ¬ 
ing,  so  only  preliminary  results  are 
presented  here. 


Issue/Observation 


Mission  Context  required  to  convert  (observable) 
performance  into  understanding  of  (tangible) 
survivability,  and  (non-observable)  Effectiveness.  (SS 
#i,pg  i) 

Validation  and  Accreditation  anchored  to  often  ill 
defined  purpose.  Understanding  how  purpose  and 
outcomes  affect  decision  can  provide  essential 
information.  (SS# I,  pg  I) 

“You  must  have  a  good  understanding  of  the 
intended  purpose  of  the  M&S”  (SS  #l,pg  2) 
“Operational  context  is  key.’’V&V  must  be  anchored 
to  purpose  (Hollenbach,  pg  4) 

Need  to  anchor  across  fidelity  (Greaney) — maps  to 
aggregation/de-aggregation  issue  raised  by 
Hollenbach 

User  (application)  requirements  are  essentials  to 
successfully  implement V&V  and  ultimately  achieving 
accreditation  (Meyer) 

“You  don’t  give  a  combat  mission  to  the  equipment, 
you  give  it  to  the  warfighter.’’  (Rice) 


Resolution 


Theoretical — “Focus’’  necessary  to  wisely  invest 
finite  W&A  resources, W&A  benchmark  to 
“Purpose’’ “Purpose’’  is  implicit,  is  often  subjective 
across  disparate  stakeholders  Perhaps  wargaming 
decision  “outcomes’’  would  add  explicit,  objective 
structure  to  “purpose”  thereby  providing  “focus”  to 
W&A  “Focus”  will  shift  based  on  Acquisition  stage: 
basic  research;  concept  formulation;  risk  reduction; 
down  select;  full  rate  production  (Sheehan,  Summary 
Slide  #1,  pg  2) 

Education — Train  people  to  build  robust 
requirements 
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Figure  I.  Impact  of  Mission  Context  on  Requirements  (names  in  parenthesis  refer  to  the  presenter  from  whose  presentation  the  comment 
was  drawn). 


Issue/Observation 


Resolution 


Sensitivity  analysis  is  invaluable  for  both  model  ND 
data  accuracy  requirements  (Fraedrich) 
Understanding  the  impact  of  parameter  sensitivity 
for  each  type  of  test/analysis  under  consideration  is 
vital  (Shepherd,  pg.  1 5) 

Analysis  of  the  impact  of  input  parameter  fidelity 
and  input  parameter  absolute  accuracy  on 
engagement  outcome  is  REQUIRED  to  enable 
credible  IRCM  effectiveness  assessment.  (Hatfield, 
pg- |o) 

Sensitivity  analysis  enables  V&V  to  be  focused  on 
the  critical  issues,  the  high  risk  factors  identified  in 
the  risk  assessment  (Fraedrich). 


Implementation — The  high  fidelity  physics-based 
modeling  approach  structures  and  supports 
sensitivity  analysis,  focuses  and  maximizes  value  of 
experiments  and  tests. 

Education — 

-  Training  in  statistical  analysis 

-  Training  to  avoid  operator  errors 

-  Documentation 

Formal  manuals  (technical,  procedural,  user, 
tutorial)  and  release  notes 

-  On-call  tech  support 

-  Users  group 

Conclusion — There  needs  to  be  a  better 
understanding  of  the  mathematical  underpinnings  of 
sensitivity  analysis  and  how  it  is  applied  to  M&S 
assessment. 


Figure  2.  The  Importance  of  Sensitivity  Analysis  to  the  Assessment  of  Simulation  Credibility 


Workshop  Accomplishments 

To  date,  the  workshop  and  follow-on 
WEAC  efforts  have  resulted  in  the  follow¬ 
ing  accomplishments — 

•  Captured  key  issues 

•  Shared  practical  approaches  to 
W&A  challenges 

•  Shared  demonstrated  and  pro¬ 
posed  solutions 

•  Workshop  attendees  synergisti- 
cally  derived  solutions 

•  Synthesized  proposed  approaches 

The  important  issues  for  M&S  credi¬ 
bility  were  captured  during  the  work¬ 
shop,  particularly  common  challenges 
that  were  identified  by  numerous 
attendees  from  multiple  organiza¬ 
tions.  For  each  of  these  issues,  the 
observations  from  multiple  presen¬ 
ters,  panel  members,  and  audience 
members  were  recorded  and  com¬ 
bined  under  a  single  heading  for  the 
issue  being  considered.  The  WEAC 
then  developed  “theoretical”  and/or 
“implemented”  solutions  to  the  issue, 
along  with  conclusions,  recommenda¬ 
tions,  and  recommendations  especial¬ 
ly  for  education  requirements  as  part 
of  the  solution.  Examples  of  the 
issues,  observations,  conclusions,  and 
recommendations  are  given  in  the 
“issues”  section  below. 


The  Issues 

The  key  issues  that  were  captured  at 
the  workshop  include — 

•  Impact  of  mission  context  on 
requirements  definition 

•  Relationship  between  correla¬ 
tion  and  validation 

•  Defining  accreditation  (good 
enough)  and  how  to  achieve  it 

•  Importance  of  sensitivity  analy¬ 
sis  to  the  assessment  of  simula¬ 
tion  credibility 

•  Critical  requirement  to  improve 
M&S  documentation 

•  Appreciate  the  importance  of 
configuration  management 

•  Mapping  M&S  use  to  the 
Verification,  Validation,  and 
Accreditation  (VV&A)  spiral 
development  process 

•  Discipline  to  the  M&S/W&A 
process 

Examples  of  the  substantiating  infor¬ 
mation  used  to  derive  the  key  issues 
can  be  found  in  Figures  1  and  2. 

While  some  of  the  key  issues  are  well 
known  to  the  M&S  community,  oth¬ 
ers  are  less  recognizable  but  no  less 
important.  Two  issues  that  are  often 
identified  as  primary  obstacles  to  the 


credible  use  of  M&S  are  the  lack  of 
well  articulated  requirements  and  a 
limited  understanding  of  the  intend¬ 
ed  use  and  associated  mission. 
Throughout  the  duration  of  the 
workshop,  several  presenters  high¬ 
lighted  the  pitfalls  associated  with  ill- 
defined  or  vague  user  requirements. 
It  was  noted  that  to  effectively  and 
efficiently  validate  and  accredit  a 
model  or  simulation,  you  must  clear¬ 
ly  understand  and  document  the 
intended  use  of  the  model  and  the 
requirements  of  the  user’s  applica¬ 
tion.  The  acceptability  criteria,  which 
flow  from  the  M&S  requirements, 
will  be  the  criteria  used  to  accredit. 
Clearly  defined  user  requirements 
will  focus  any  necessary  V&V  activi¬ 
ty,  thereby  making  the  accreditation 
process  more  affordable  and  practi¬ 
cal.  Figure  1  (see  page  8)  illustrates 
the  WEAC  findings  associated  with 
this  issue. 

When  it  comes  to  M&S  credibility, 
several  presenters  raised  the  question 
of  defining  “good  enough,”  identify¬ 
ing  it  as  a  critical  factor  in  establishing 
the  required  levels  of  V&V.  While  in 
most  cases,  it  is  unrealistic  to  expect 
something  to  be  validated  100%,  it  is 
essential  that  the  user  balances  the 
risks  being  assumed  in  using  the  results 
of  the  M&S  for  a  specific  application 
against  the  cost  of  accumulating  V&V 
evidence.  A  quote  from  WS  I 
“Balancing  the  cost  of  knowing 
against  the  risk  of  assuming,”  really 
cuts  to  the  heart  of  the  matter. 
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Representation 
&  Behavior 
Drivers 


Mission  Context 
provides  authoritative  info 
about  domain 


Requirements 
defines  what  M&S  needs  to  do 
and  what’s  extraneous 


Process 

Focus  on 
Representation  & 
Behaviors 


Process 

Drivers 


Point  to  Point 
Correlation 

Methodology  to  compare 
observed  to  expected 

SMEs  assess  correlation 

Sensitivity  Analysis  used  to 
determine  the  impact  of  correlation 
variance  on  simulation  results 


Validation 

Correlation  of  sim  results  to  a  real  world 
benchmark  (referent) 

SMEs 

1 .  Develop  referent 

2.  Assess/compare  M&S  results  against  referent 
(test  data,  SME  info,  etc.) 

3.  Data  collection  plan  and  design  of  experiments 

Sensitivity  Analysis 

I  Vrsi  ld  sress  M&S  and  data  accuracy 


2.  Izzi  aiiiwvity  of  M&S  variables  &  parameters 

3  Ai n  nM  world  processes  reaction  to  change 

4  L:-:d  ix*  plin  testing  that  is  focused  and  cost- 
effective 


Data  populates  M&S  (measurement  and  collection) 

Documentation  captures  and  maintains  history  of  M&S  and  Data 
Configuration  Management  tracks  versions  of  and  changes  to  M&S 


Good  Enough 

Correlation  of  M&S  capabilities  to 
M&S  requirements;  assessment  of 
M&S  adequacy  for  purpose 

SMEs  evaluate  how  well  M&S 
meets  acceptability  criteria 

Sensitivity  Analysis  used  to 
determine  the  impact  of 
assumed  risks  with  acceptance 
of  simulation  results 


Figure  3.  Conceptual  Framework  for  Relating  Issues 
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Multiple  references  were  made  to  the 
employment  of  subject  matter 
experts  (SMEs)  in  defining  “good 
enough”  for  a  particular  intended 
use.  It  is  appropriate  in  some 
instances  (i.e.,  when  evaluating  the 
representation  of  future  systems)  for 
the  SME  to  play  a  key  role  in  assess¬ 
ing  credibility  when  relevant  valida¬ 
tion  data  may  not  exist.  However,  it 
should  be  noted  that  the  qualitative 
nature  of  SME  assessment  would  not 
be  appropriate  in  all  cases.  Some 
instances  may  require  a  more  quanti¬ 
tative  evaluation.  One  such  quantita¬ 
tive  assessment,  sensitivity  analysis, 
was  mentioned  quite  often  in  the 
course  of  the  workshop.  Sensitivity 
analysis  used  to  support  the  estab¬ 
lishment  of  M&S  credibility  can  pro¬ 
vide  a  wealth  of  information  about 
the  model  or  simulation  as  well  as 
information  of  great  value  to  SME 
assessments.  Figure  2  (see  page  9)  is 
another  illustration  of  the  WSII  out¬ 
put  to  date  that  lists  the  observations 
and  recommendations  from  the 
WEAC  with  respect  to  sensitivity 
analysis. 

There  was  considerable  focus  during 
the  workshop  on  the  elements  of 
good  model  management  and  docu¬ 
mentation,  and  the  impact  these 
processes  have  on  enhanced  M&S 
credibility.  The  value  of  requirements 


management  has  already  been 
addressed.  However,  good  configura¬ 
tion  management  (CM)  is  also  an 
essential  part  of  the  equation.  Several 
presenters  highlighted  that  the  posi¬ 
tive  relationship  between  model 
management  processes  and  the  qual¬ 
ity  of  model  results  demonstrates  the 
value  of  applying  the  capabilities 
maturity  model  (CMM)  to  support 
the  establishment  of  M&S  credibility 
by  evaluating  the  quality  of  a  simula¬ 
tion’s  underlying  software  develop¬ 
ment  process. 

Another  common  thread  running 
through  the  workshop  briefings  and 
panel  discussions  was  the  need  to  find 
more  mechanisms  to  share  informa¬ 
tion  and  experiences.  This  is  especial¬ 
ly  important  for  cash-poor  programs 
that  would  benefit  from  capitalizing 
on  or  leveraging  the  V&V  efforts  of 
others.  Forums  such  as  this  workshop 
and  individual  model  user  group 
meetings  facilitate  this  exchange  of 
information.  Good  documentation  by 
model  developers  will  also  assist  in  a 
program’s  reuse  of  legacy  models, 
providing  information  on  previous 
V&V  results  that  could  serve  as  the 
foundation  for  new  V&V  activities. 
DOT&E  stressed  the  need  to  improve 
model  documentation,  especially  in 
identifying  limitations  and  assump¬ 
tions,  as  important  to  reducing  the 


cost  and  time  associated  in  V&V 
efforts. 

While  each  of  the  key  issues  discussed 
above  stands  alone,  they  are  all  relat¬ 
ed.  Figure  3  demonstrates  the  relation¬ 
ship  among  these  key  technical  issues. 
The  diagram  has  three  levels.  The  first 
level  is  focused  on  the  definition  of  the 
Mission  Context  from  which  the 
Requirements  are  defined.  These  two 
items,  Mission  Context  and 
Requirements,  will  drive  the  required 
representations  and  behaviors.  The 
second  level  presents  the  process 
through  which  the  representations  and 
behaviors  will  be  assessed  for  credibil¬ 
ity.  The  third  level  shows  the  key 
process  drivers  [e.g.,  data,  documenta¬ 
tion,  and  configuration  management 
(CM)],  which  play  a  critical  role  in  the 
assessment  and  execution  process. 
Under  the  concepts  of  Point  to  Point 
Correlation,  Validation,  and  Good 
Enough,  the  role  that  SMEs  and 
Sensitivity  Analysis  would  play  has 
been  emphasized.  However,  these  are 
not  the  only  activities  that  will  take 
place  as  these  processes  are  imple¬ 
mented.  The  arrows  accompanying 
the  process  drivers  (data,  documenta¬ 
tion,  and  CM)  indicate  that  they 
impact  the  whole  assessment  process. 
They  also  apply  to  the  definition  of  the 
Mission  Context  and  Requirements. 


The  Future  of  the  WEAC  and 
the  M&S  Credibility 
Workshops 

During  Workshop  II,  both  DOT&E 
and  the  JTCG/AS  raised  questions 
regarding  the  role  and  sponsorship  of 
the  M&S  Credibility  Workshops. 
While  they  were  the  primary  sponsors 
for  the  first  two  workshops,  it  was 
emphasized  that  in  order  to  procure 
their  continued  sponsorship, 
Workshop  II  must  result  in  a  product 
of  real  value.  In  response  to  that  chal¬ 
lenge,  the  WEAC  is  working  to  devel¬ 
op  tow  products:  The  WS  II  outbrief 
and  an  associated  map  which  identi¬ 
fies  post- Workshop  II  activities.  Addi¬ 
tionally,  the  WEAC  is  working  to 
establish  partnerships  with  other 
organizations  that  have  vested  inter¬ 
ests  in  M&S  credibility  (e.g.,  DMSO, 
the  Service  Offices).  These  partner¬ 
ships  will  serve  to  facilitate  imple¬ 
mentation  of  the  activities  identified 
in  the  roadmap. 

Finally,  numerous  comments  from 
attendees  regarding  the  valuable  role 
that  follow-on  M&S  credibility 
workshops  was  addressed.  With  the 
importance  placed  on  M&S  in  the 
systems  acquisition  process,  this  type 
of  forum  is  vital  by  providing  a 
mechanism  for  the  exchange  of  ideas 
and  lessons  learned  among  profes¬ 
sionals  in  the  field.  However,  the 
WEAC  has  recognized  that  changes 
should  be  made  to  increase  the  value 
of  the  workshops.  As  mentioned 
above,  the  WEAC  recognizes  the 
importance  given  to  education  in  the 
solutions  promoted  at  WS  II. 
Therefore,  the  WEAC  has  proposed 
that  any  follow-on  activity  focus  on 
these  education  requirements  in 
M&S  credibility. 

While  education  was  always  one  of 
the  objectives  of  the  workshop, 
members  of  the  WEAC  are  consider¬ 
ing  specific  proposals  that  will 
increase  the  focus  on  W&A  related 
training.  These  proposals  include 
conducting  smaller  and  more  focused 
seminars  and  providing  mechanisms 
for  writing,  publishing  and  dissemi¬ 
nating  white  papers,  and  articles  on 
M&S  credibility  topics,  such  as — 

•  Configuration  management 


Requirements  management 
Model  documentation 


•  Risk  assessments  and  mitigation 

•  Sensitivity  analysis 

Therefore,  while  the  M&S  Credibility 
Workshop  II  has  ended,  the  forward 
progress  continues.  The  WEAC  is 
moving  ahead  with  the  ideas  gleaned 
from  the  presenters  and  panels  in 
Reno.  A  roadmap  to  more  credible 
M&S  is  being  developed  and  partner¬ 
ships  are  being  formed  to  increase  the 
likelihood  of  its  success.  ♦ 

Mr.  Ron  Ketcham  has  a  B.A.  degree  from 
Transylvania  University  ( Lexington , 
Kentucky)  in  Economics ,  and  a  M.5.  in 
Economics  and  Industrial  Engineering  from 
Iowa  State  University.  He  is  currently  head 
of  the  System  Survivability  Integration 
Branch  at  the  Naval  Air  Warfare  Center 
Weapons  Division  (NAWCWD  Code 
418100D).  He  is  also  the  Chairman  of  the 
Survivability  Assessment  Subgroup  for  the 
JTCG/AS. 

Ms.  SimoneYoungblood  is  a  member  of  the 
Principal  Professional  Staff  at  the  Johns 
Hopkins  University  Applied  Physics  Labor¬ 
atory  (JHU/APL).  For  the  past  five  and  a  half 
years ,  Ms.  Youngblood  has  served  as  the 
DoD  VV&A  focal  point  in  her  position  as 
DMSO's  VV&A  Technical  Director. 
Leveraging  an  extensive  background  in 
simulation  development,  modification  and 
application,  M  s.Youngblood  has  been  active 
in  the  VV&A  community  for  the  past  10 
years.  She  contributed  to  the  development 
of  Service  and  DoD  policy  and  guidance. 
She  has  also  led  or  supported  a  variety  of 
VV&A  efforts.  From  1997  to  the  present, 
M s.Youngblood  has  served  as  the  chair  of 
the  S/1/1/  VV&A  Forum.  M s.Youngblood  is 
also  active  in  the  Society  for  Computer 
Simulation  and  the  Military  Operations 
Research  Society.  Ms.  Youngblood  has  a 
B.A.  in  Mathematics  as  well  as  a  B.S.  and 
an  M  .S.  in  Computer  Science. 
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Scientific  results  cannot  be  used 
effectively  by  soldiers  who  have 
no  understanding  of  them,  and 
scientists  cannot  produce  results  use¬ 
ful  for  warfare  without  an  under¬ 
standing  of  the  operations. 

— Dr  Theodore  Von  Karman,  Chairman, 
Scientific  Advisory  Group,  1944 

“Five  seconds ,  bomb  doors — open, 
three,  two,  one,  internals  are  going, 
internals  are  gone,  externals  are 
going,  whomp,  externals  are  gone, 
bomb  doors — closed,  pilot,  come 
right  to  heading  164,  known  SA-2 
site  2  o’clock  25  miles.  What  was 
that  whomp?”  So  went  the  first  com¬ 
bat  sortie  of  a  young  (and  certainly 
less  gray)  Lieutenant  Mann  in  the 
hostile  skies  of  Iraq.  Survivability 
engineers  by  definition  care  about 
aircraft  survivability,  but  trust  me,  at 
that  point  I  really  cared  about  air¬ 
craft  survivability. 

But,  back  to  that  unexpected 
whomp... as  we  dropped  our  full 
load  of  24  CBU-87  combined  effects 
munitions  they  prematurely  func¬ 


tioned,  filling  the  air  with  4,848 
armor  piercing,  fragmenting,  and 
incendiary  BLU-97  submunitions. 
We  had  dodged  SA-2  missiles  and 
100  mm  anti-aircraft  fire  on  the  way 
to  the  target  only  to  take  hits  from 
our  own  weapons  detonating  close 
below  the  aircraft.  The  Republican 
Guard’s  Medina  Division  would  keep 
its  tanks  another  day. 

Operators  have  two  goals  in  mind 
when  they  plan  a  combat  mission. 
First  and  always,  plan  to  accomplish 
the  mission.  The  mission  is  para¬ 
mount,  be  it  iron  and  fire  on  the 
heads  of  bad  guys,  supplies  to  an  air¬ 
field,  or  passing  gas  to  other  aircraft. 
Second,  plan  to  survive.  Crews  need 
to  return  an  aircraft  and  crew  that 
can  fly  the  next  mission.  In  summary, 
the  products  of  the  mission  planner 
are  live  crews  and  dead  targets.  In 
this,  aviators  and  those  working  on 
aircraft  survivability  improvements 
share  a  common  goal,  although  we 
pursue  it  in  different  ways. 

Combat  mission  planners  focus 
almost  entirely  on  decreasing  the  sus¬ 


Figure  2.  CBU-87  deploying  its  BLU-97 
submunitions. 


ceptibility  of  their  aircraft.  It  might 
seem  like  common  sense  but  avoiding 
a  hit  is  always  the  goal.  Especially 
since  one  of  the  critical  components  of 
that  aircraft  is  a  warm  pink  body,  very 
often  belonging  to  the  person  that 
planned  the  mission.  As  Dr.  Bob  Ball, 
the  guru  of  aircraft  survivability  edu¬ 
cation  notes  in  his  classic  text,  “When 
penetrated,  crew  members  tend  to  lose 
their  ability  to  function.”  Indeed! 

I  wonder  to  what  degree  the  engi¬ 
neers  working  to  improve  aircraft 
survivability  know  how  aircrews 
work  to  reduce  susceptibility,  and 
vice  versa?  It  would  be  pretty  diffi¬ 
cult  to  keep  up  with  even  one  specif¬ 
ic  aircraft.  Operators  never  stop 
changing  tactics,  techniques,  and 
procedures.  As  a  flyer,  I  believe  that  it 
is  extremely  important  that  as  we 
change  our  tactics,  we  take  into 
account  whatever  survivability  meas¬ 
ures  have  been  built  into  the  aircraft. 
We  certainly  don’t  want  to  develop  a 
new  threat  avoidance  measure  that 
negates  other  aspects  of  our  surviv¬ 
ability,  but  of  course,  that  street  runs 
both  ways. 

Survivability  enhancements  that  we 
can’t  or  won’t  use  because  they  are  at 
odds  with  how  we  employ  the  aircraft 
are  not  very  useful.  As  an  example,  I 
once  heard  a  survivability  engineer 


Figure  I.A  B— 52  returns  from  its  bombing  mission  in  support  of  the  USCENTCOM  execu¬ 
tion  of  Operation  Enduring  Freedom,  March  23, 2002. 


state  with  conviction  that  aircrews 
operate  stupidly  by  not  simply  drop¬ 
ping  flares  preemptively  in  anticipa¬ 
tion  of  a  MANPADS  launch.  His 
organization  had  designed  flares 
thinking  that  we  would  use  them  that 
way.  They  were  not  aware  of  several 
problems  with  this  idea.  First,  most 
aircrew  will  never  use  flares  preemp¬ 
tively  at  night  because  they  will  high¬ 
light  the  aircraft  to  a  myriad  of  other 
threats.  Second,  because  many  air¬ 
craft  have  a  limited  number  of  flares, 
they  cannot  be  used  on  every  pass. 
Third,  preemptive  flaring  could  very 
well  exhaust  the  entire  flare  load, 
leaving  no  flares  available  for  reactive 
use  (i.e.,  after  a  shot  has  been  fired). 


Figure  3.  Three  B-52G  Stratofortress  aircraft  leave  clouds  of  exhaust  behind  as  they  take 
off  in  formation. 


So  with  all  that  said,  what  is  the 
point?  Speaking  as  an  aviator,  I 
believe  that  it  is  vitally  important 
that  the  engineering  community 
know  what  we  do,  and  that  we  in 
turn,  know  why  and  what  they  have 
done  to  our  aircraft.  A  large  amount 
of  the  readers  of  this  newsletter  most 
likely  follow  the  traditional  break¬ 
down  of  the  aircraft  survivability  dis¬ 
cipline  into  susceptibility  and  vulner¬ 
ability  components.  I  would  hold 
that  a  vital  third  component — one 
that  intricately  intertwines  conven¬ 
tional  survivability — is  the  operator. 
It  is  the  operator  that  ultimately 
takes  the  work  of  the  rest  and  goes 
into  harms’  way. 

For  example,  consider  aircraft  vulner¬ 
ability.  In  many  cases,  the  various  vul¬ 
nerability  reduction  features  that  have 
been  built  into  or  added  onto  the  air¬ 
craft  are  nearly  transparent  to  the 
operator.  Most  of  the  time  this  is  not 
a  problem — vulnerability  reduction 
measures  tend  to  be  passive  in  nature, 
requiring  little  to  no  interaction  with 
the  aircrew.  But  in  a  combat  situation, 
especially  after  a  shot  has  been  fired, 
knowledge  and  understanding  of 
these  features  could  literally  mean  the 
difference  between  death  and  a  really 
good  “there  I  was”  story  for  the  bar. 
Just  about  every  aircraft  flown  has 
something  called  a  “last  ditch” 
maneuver  that  is  initiated  when  a  hit 
is  imminent.  In  this  situation,  would¬ 
n’t  it  be  useful  for  the  crew  to  know 
which  parts  of  the  aircraft  are  better 
able  to  withstand  a  hit,  or  which  sys¬ 
tems  are  disabled  or  destroyed  after  a 
hit  in  a  particular  location?  And 


wouldn’t  it  be  equally  useful  for  the 
vulnerability  reduction  engineers  to 
have  some  knowledge  about  how  the 
aircraft  is  employed  during  those  last 
critical  3-5  seconds  when  the  threat  is 
inbound  to  the  aircraft? 

When  going  into  “Indian  country” 
operators  put  a  lot  of  faith  into  the 
work  that  the  two  survivability  com¬ 
munities  have  accomplished.  While 
aircraft  survivability  has  most  defi¬ 
nitely  improved  since  the  enormous 
attrition  rates  of  Vietnam  and  earlier 
conflicts,  it  would  certainly  lead  to  a 
better  product  if  engineers  and  oper¬ 
ators  sought  out  each  other  and 
cooperated  on  issues  before  mixing  it 
up  with  the  bad  guys.  An  engineer 
working  on  aircraft  survivability  that 
never  talks  to  an  operator  is  missing 
out  on  a  deep  well  of  information 
that  could  make  his  or  her  work 
more  useful  in  the  field.  Survivability 
programs  need  to  be  collaborative 
efforts  between  engineers  and 
warfighters.  Operators  would  better 
understand  the  protective  measures 
and  tools  that  are  on  the  aircraft,  and 
engineers  would  better  understand 
what  operators  intend  to  do  with 
their  efforts  in  combat.  ♦ 

It  is  not  sensible  for  the  ones  who  spe¬ 
cialize  at  the  other  end ,  to  have  such 
disregard  for  each  other.  (They  don't 
actually  but  people  say  they  do.) 

— Dr  Richard  Feynman,  Nobel  Prize 
Winning  Physicist,  raconteur,  bongo  player 


Major  Robert  W.  Mann  is  a  B-52 
Instructor  Radar  Navigator  and  Weapons 
Officer  with  over  3,000  hours  in  the 
B-52G,  B-52H,  RF-4C,  T-43,  E-3A,  and 
T-37  including  125  combat  hours.  He 
took  part  in  Desert  Storm, Allied  Force,  and 
is  currently  deployed  in  support  of  opera¬ 
tion  Enduring  Freedom.  He  has  a  B.A.  in 
History  from  The  Citadel,  an  M.S.  in 
Management  from  Embry-Riddle 
University,  and  an  M.S.  in  Military 
Operational  Arts  and  Sciences  from  The  Air 
University.  He  may  be  reached  at 
robert.mann@  b2.wpafb.af.mil. 


Figure  4.  An  Air  Force  munitions  specialist 
from  the  28th  Air  Expeditionary  Wing 
changes  the  configuration  of  a  B-52 
external  bomb  rack  from  Joint  Direct 
Attack  Munitions  to  conventional  muni¬ 
tions,  November  28, 2001,  during 
Operation  Enduring  Freedom. 
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The  PUSH  for  an 

Advanced  Joint  Effectiveness  Model  (AJEM) 

■  by  M r.  Doug  McCown,  Mn  Ron  Thompson,  and  Mn  Michael  RWeisenbach 


In  the  early  1990s,  the  Anti-air 
Working  Group  of  the  Joint 
Technical  Coordinating  Group 
for  Munitions  Effectiveness 
(JTCG/ME)  identified  the  need  for  a 
new  anti-air  effectiveness  model.  In 
1993,  a  group  led  by  Mr.  Bruce 
Nofrey  of  the  Naval  Air  Warfare 
Center,  Weapons  Division  at  Port 
Mugu,  California,  and  Mr.  Tom 
Wasmund  of  the  Naval  Surface 
Warfare  Center,  Dahlgren,  Virginia 
(both  now  retired)  proposed  this 
development  to  the  Defense  Modeling 
and  Simulation  Office  (DMSO). 
DMSO  encouraged  the  development 
of  the  Advanced  Joint  Effectiveness 
Model  (AJEM)  but  stipulated  that  it 
made  use  of  the  Modular  UNIX™- 
based  Vulnerability  Estimation  Suite 
(MUVES)  environment  developed  by 
the  Army  Research  Laboratory 
(ARL).  At  the  same  time,  the 


JTCG/ME  received  critical  support 
for  AJEM  development  from  the  Joint 
Technical  Coordinating  Group  on 
Aircraft  Survivability  (JTCG/AS). 
These  coordinated  efforts  resulted  in 
the  1997  publication  of  the  Software 
Requirements  Specification  for  the 
AJEM.  With  the  incorporation  of 
MUVES,  AJEM  now  possessed  the 
potential  to  become  the  standard  tool 
of  the  Department  of  Defense  (DoD) 
for  evaluating  the  vulnerability  of  air¬ 
craft,  missiles,  and  ground-mobile  tar¬ 
gets,  as  well  as  the  lethality  and  effec¬ 
tiveness  of  munitions. 

Spiraling  in  on  the  goal  to  convert 
that  potential  to  reality,  there  have 
been  four  releases  of  AJEM. 

•  September  2000  AJEM  1.0 — 
Provided  basic  functionality 


•  August  2001  AJEM  1.1— 
Improved  functionality,  included 
MUVES  upgrades  and 
Encounter  Visualization  (EVT) 

•  November  2001  AJEM  1.2 — 
Addressed  user  identified  defi¬ 
ciencies 

•  June  2002  AJEM  2.0 — Provided 
upgrades  to  the  Graphical  User 
Interface  and  included 
BRL-CAD™  6.0 

As  shown  in  Figure  1,  the  vulnerabil¬ 
ity  and  lethality  (V/L)  processes  cur¬ 
rently  employed  by  the  survivability 
and  lethality  communities  can 
involve  a  variety  of  tools  in  a  variety 
of  combinations.  These  tools  can  be 
sorted  into  three  basic  functions:  tar¬ 
get  geometry,  vulnerability  assess¬ 
ment,  and  endgame.  AJEM  is 
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The  goal  of  AJEM  is  to  perform  all  these  functions 
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Figure  I.  Common  Vulnerability/Lethality  Processes  and  Tools 


designed  to  perform  all  of  these  func¬ 
tions  in  a  single  software  package. 

Making  AJEM  the  standard  DoD 
tool  to  perform  all  of  these  vulnera¬ 
bility  and  lethality  functions  is  going 
to  take  a  bit  of  a  PUSH.  That  PUSH 
has  taken  the  form  of  FY02  and  03 
investments  in — 

•  Production  reliability 

•  Usability 

•  Suitability  definitions,  and 

•  Handling  Verification,  Validation 
and  Accreditation  (W&A)  issues. 

AJEM  is  intended  to  be  a  production 
analysis  tool.  AJEM  production  reli¬ 
ability  means  a  broad  spectrum  of 
users  should  be  able  to  employ 
AJEM  reliably,  rapidly,  and  repeti¬ 
tively.  AJEM  has  to  meet  the  needs  of 
this  broad  spectrum  of  users  if  it  is 
going  to  be  the  standard  tool  for 
these  applications. 

That  ushers  in  the  second  component 
of  the  PUSH — usability.  For  usability, 
AJEM  documentation  will  be  contin¬ 
uously  maintained  to  ensure  accura¬ 
cy  and  clarity.  Example  applications 
for  common  AJEM  applications  will 
be  provided  (i.e.,  LFT&E,  missile 
flight  test  analyses,  design  trades  for 


aircraft,  and  warhead  applications 
are  being  considered).  Also,  the 
JTCG/ME  and  JTCG/AS  have  spon¬ 
sored  several  introductory  training 
courses  to  address  analysts’  AJEM 
usability  issues.  We  are  focusing  in 
FY02  and  03  on  fully  supporting  the 
AJEM  user  community  by  way  of  a 
solid  foundation  of  support  for  sev¬ 
eral  production  analyses  that  will 
demonstrate  both  AJEM  usability 
and  capability.  This  support  has  and 
will  help  new  users  with  their  specif¬ 
ic  AJEM  applications. 

The  third  component  of  this  PUSH  is 
suitability.  AJEM  and  its  underlying 
structure,  Modular  UNIX™-based 
Vulnerability  Estimation  Suite 
(MUVES),  has  been  shown  by  ARL 
to  be  suitable  for  a  variety  of  ground- 
mobile  V/L  applications.  The 
JTCG/AS  and  JTCG/ME  are  current¬ 
ly  engaged  in  establishing  and 
demonstrating  the  suitability  of 
AJEM  for  the  needs  of  the  air  target 
community  through  the  above  men¬ 
tioned  Production  Analyses,  and 
through  the  development  of  air  tar¬ 
get  specific  test  cases.  The  ARL  has 
significantly  improved  the  suitability 
of  AJEM  for  air  targets  by  incorpo¬ 
rating  FASTGEN  geometric  model¬ 
ing  primitives  into  BRL-CAD™6.0. 

The  final  element  in  the  PUSH  for 
AJEM  is  in  handling  W&A.  This 


W&A  process  is  being  accomplished 
in  accordance  with,  Procedures  for 
Accreditation  of  Software  by  the 
JTCG/ME  (61  JTCG/ME-1-7).  The 
AJEM  W&A  process  is  also  in 
accord  with  the  JTCG/AS  process  as 
developed  by  the  joint  accreditation 
support  activity  (JASA).  The  expect¬ 
ed  result  is  the  preparation  of  one  or 
more  Accreditation  Support 
Packages  (ASP)  for  AJEM.  These 
AJEM  ASPs  will  be  based  both  on 
results  of  current  efforts  as  well  as  on 
a  great  deal  of  existing  information. 
Current  efforts  include  comparisons 
of  AJEM  results  with  those  devel¬ 
oped  using  the  computation  of  vul¬ 
nerability  and  repair  times 
(COVART)  model,  which  is  the  cur¬ 
rent  standard  analysis  tool.  These 
comparisons  are  being  conducted  for 
both  ground-mobile  and  high  per¬ 
formance  fixed  wing  aircraft  targets. 

Existing  AJEM  documentation  con¬ 
sists  primarily  of  the  new  MUVES 
ASP-1  and  analyst  manual.  While 
the  MUVES  makes  up  a  significant 
portion  of  the  AJEM  product,  the 
existence  of  MUVES  verification  and 
validation  doesn’t  get  AJEM  com¬ 
pletely  off  the  hook.  Figure  2  high¬ 
lights  (dark  outlined  boxes)  ASP 
issues  that  are  unique  to  AJEM. 

The  ASP  will  be  submitted  to  an 
AJEM  Model  Review  Committee 


Figure  2.  Accreditation  Support  Package  (ASP)  Issues 
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(MRC)  that  is  currently  being 
planned  for  FY03.  The  objective  of 
the  MRC  will  be  to  make  an  AJEM 
accreditation  recommendation  con¬ 
cerning  the  use  of  AJEM  by  the 
JTCG/ME  to  develop  estimates  for 
munitions  effectiveness  [Joint 
Munitions  Effectiveness  Manuals 
(JMEM)].  The  MRC  will  have  the 
responsibility  to  make  sure  that 
AJEM  is  applicable  and  sufficiently 
credible  for  the  JMEM  application. 
In  this  process,  the  MRC  will  use  the 
AJEM  ASP,  along  with  information 
on  the  AJEM  software  configuration 
control  process,  and  AJEM  manage¬ 
ment  support  structure.  Our  intent  is 
then  to  make  the  ASP,  along  with 
other  pertinent  accreditation  docu¬ 
mentation  available  as  a  resource  for 
others  with  a  need  to  accredit  AJEM 
for  their  own  specific  applications. 
The  JTCG/AS  develops  ASP  docu¬ 
ments  for  the  major  M&S  supported 
by  that  organization  and  distributes 
them  to  users  through  the 
Survivability /Vulnerability  Inform¬ 
ation  Analysis  Center  (SURVIAC). 
AJEM  will  become  one  of  those 
M&S  supported  by  the  JTCG/AS,  as 
well  as  the  JTCG/ME. 

To  learn  more  about  AJEM  or  pro¬ 
vide  us  with  valuable  feedback  to 
help  us  improve  the  quality  of  the 
AJEM  product,  visit  our  Web  site  at 
http://www.ajem.com  ♦ 


Mr.  Doug  McCown  was  appointed  AJEM 
model  manager  in  November  of  2001.  He 
received  his  B.5.  in  Systems  Science  from 
the  University  of  West  Florida.  H  e  has  been 
involved  in  the  JTCG/ME  for  15  years.  He 
has  19  years  of  experience  in  vulnerability, 
lethality,  testing,  technology  development, 
and  weapon  system  development.  M  ost  of 
this  work  has  been  for  air-to-air  weapon 
systems.  He  has  been  the  principal  main- 
tainer  of  the  missile  lethality  code,  Shazam, 
at  Eglin  Air  Force  Base,  delivering  Wright- 
Patterson  their  first  copy  in  1986.  He  may 
be  reached  at  850.882.9585,  or 
mccown@  eglin.af.mil. 

In  September  2001,  Mr  Ronald  A. 
Thompson  Joined  the  Joint  Technical 
Coordinating  Group  for  Munitions 
Effectiveness  (JTCG/ME)  Program  Office  at 
Aberdeen  Proving  Ground,  Maryland.  He  is 
the  Anti-Air  Technical  Program  M  anager, 
AJEM  coordinator  and  Contracting  Officers 
Technical  Representative.  Prior  work  since 
1983  at  the  Army  Materiel  Systems 
Analysis  Activity  (AMSAA)  was  in  ground 
combat  vehicle  system  development, 
methodology  development,  test  and  evalua¬ 
tion,  and  live  fire.  M r.Thompson  has  a  B.S. 
in  Mechanical  Engineering  from  the 
University  of  Maryland,  is  a  Registered 
Professional  Engineer  and  has  an  M.S.  in 
Technical  Management  from  the  Johns 
Hopkins  University. 


M  r.Weisenbach  is  currently  the  Air  Force  civil¬ 
ian  representative  in  the  Joint  Technical 
Coordinating  Group  on  Aircraft  Survivability 
(JTCG/AS)  Central  Office.  He  has  been  in  the 
Central  Office  since  May  2000.  His  duties 
include  providing  oversight  and  direction  to 
the  Survivability  Assessment  Subgroup.  Prior 
to  his  current  assignment,  Mr.  Weisenbach 
spent  14  years  at  the  Aeronautical  Systems 
Center,  W  right-Patterson  AFB,  Ohio,  in  the 
Mission  Analysis  and  Engineering 
Directorates.  He  supported  the  survivability 
and  effectiveness  analyses  of  conceptual 
designs  ranging  from  advanced  tactical  trans¬ 
ports  to  cruise  missile  defense  systems,  and 
also  served  as  the  F-22  Live  Fire  Test 
Engineer  for  12  months.  He  is  a  1984  grad¬ 
uate  of  the  University  of  Notre  Dame  with  a 
B.S.  is  Aerospace  Engineering.  He  can  be 
reached  at  703.607.3509,  ext.18, 
weisenbachm@  navair.navy.mi 


Successful  Correlation  of  M&S  to 
Live  Fire  Test  Results 

for  the  U.S.  Army’s  SIIRCM  Program 


■  by  Mn  James  Hatfield  III,  Dn  Mike  Neer;  and  Ms.  Julie  Locker 


With  the  war  on  ter¬ 
rorism,  there  is  a 
new  urgency  to 
develop  and  field 
self  protection  systems  for  U.S.  Army 
aircraft.  The  U.S  Army’s  Suite  of 
Integrated  Infrared  Countermeasures 
(SIIRCM)  has  completed  a  successful 
integrated  test  program  in  CY2001- 
CY2002  and  is  currently  entering 
into  production  for  fielding  on  the 
MH-47  Chinook  helicopter.  The 
SIIRCM  system  is  developed  and 
managed  by  the  U.S.  Army’s  Program 
Manager  for  Aviation  Electronic 
Systems  located  at  Redstone  Arsenal, 
Alabama.  The  system  is  designed  to 
provide  aircraft  defensive  counter¬ 
measures  for  defeating  incoming 
heat-seeking  missile  threats  to  U.S. 
Army  attack,  utility,  cargo,  and  spe¬ 
cial  operations  aircraft.  The  SIIRCM 
system  components  include  the 
Advanced  Threat  Infrared 


Countermeasures  (ATIRCM), 
Common  Missile  Warning  System 
(CMWS),  and  the  Advanced  Infrared 
Countermeasure  Munitions  (AIR- 
CMM)  (see  Figure  1).  The  U.S. 
Army’s  current  IRCM  configuration 
for  the  fleet  helicopter  consists  of  the 
Countermeasures  Set  (AN/ALQ- 
144 A)  for  the  AH-64  Apache  and 
the  UH/MH-60  Blackhawk,  and  the 
AN/ALQ-1 56  missile  approach 
detector  and  M-130  flare/chaff  dis¬ 
penser  for  the  CH/MH-47  Chinook. 

The  SIIRCM  program  is  a  simulation 
based  acquisition  program. 
Obviously,  live  missiles  cannot  be 
fired  at  manned  aircraft.  Therefore, 
senior  Department  of  Defense  (DoD) 
management  must  make  production 
decisions  without  ever  firing  missiles 
at  the  aircraft  they  are  trying  to  pro¬ 
tect.  Models  and  simulations  are 
used  to  overcome  test  limitations  and 


supplement  system  performance  test 
data.  One  of  the  early  uses  of  model¬ 
ing  and  simulation  (M&S)  in  the 
SIIRCM  program  was  to  make  pre¬ 
test  predictions  in  support  of  test 
plan  optimization,  risk  reduction, 
range  safety  analyses,  and  on-site 
quick  look  analysis. 

The  most  important  use  of  M&S  has 
been  to  supplement  limited  open  air 
range  test  results  in  evaluating  key 
system  performance  measures  such 
as  probability  of  declaration,  proba¬ 
bility  of  countermeasure,  and  false 
alarm  rate.  Another  major  testing 
requirement  supplemented  by  M&S 
is  the  ability  to  test  system  hardware 
and  software  modifications.  As  the 
system  goes  through  the  development 
and  test  cycle,  problems  are  found 
and  corrected  leading  to  the  need  for 
regression  testing.  Repeating  open- 
air  tests  can  be  cost  prohibitive,  so 
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Figure  I.  SIIRCM  System  Components 
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Figure  2.  SIIRCM  Simulation  Tools 


modeling  and  simulations  are  often 
the  only  affordable  way  to  test  sys¬ 
tem  upgrades. 

Specific  M&S  tools,  were  developed 
to  meet  program  requirements  (see 
Figure  2). 

A  full  digital  simulation  called  the 
End  to  End  Model  (E2E)  was 
required  to  determine  the  system  false 
alarm  rate,  probability  of  declaration, 
and  probability  of  countermeasure. 
Once  accredited  this  model  could 
produce  the  thousands  of  simulated 
missiles  and  potential  false  alarm 
sources  (PFAS)  needed  to  achieve  the 
confidence  levels  required. 

A  simulation  tool  was  also  required 
to  illuminate  the  test  aircraft  with 
missile  radiation  so  system  perform¬ 
ance  could  be  determined  when  the 
system  is  installed  on  actual  aircraft. 
To  satisfy  this  requirement  a  ground 
based  missile  simulator  called  Super 
Multi-Role  Electro-optical  Simulator 
(SMEOS)  was  developed  which 
allows  testing  of  the  SIIRCM  system 
during  tactical  flight  profiles  by  sim¬ 
ulating  missiles  from  various  ranges, 
directions,  and  in  different  weather 
conditions.  The  SMEOS  has  colli¬ 
mated  ultraviolet  (UV)  and  infrared 
(IR)  output  beams  each  of  which  are 
programmed  with  either  irradiance 
data  from  actual  missile  tests  or  irra¬ 
diance  predictions  from  the  E2E 


model.  SMEOS  also  has  an  IR 
radiometer  to  detect  jam  energy  from 
the  ATIRCM  laser.  SMEOS  has  a 
video  camera  for  verification  of  tar¬ 
get  tracking  and  witness  sensors  to 
monitor  beam  output. 

A  simulation  tool  was  also  needed  to 
test  the  system  under  stressing  condi¬ 
tions  in  which  the  CMWS  could  be 
exposed  to  an  almost  unlimited  num¬ 
ber  of  simultaneous  missile  and  PFAS 
sources.  The  System  Integration  Lab 
(SIL)  satisfied  this  requirement.  The 
SIL  contains  an  actual  CMWS 
Electronic  Control  Unit  (ECU)  with 
simulated  UV  sensor  inputs  produced 
by  the  E2E  model.  The  SIL  produces 
INS  data  for  input  to  the  ECU  simu¬ 
lating  the  aircraft  1553  data. 

The  last  simulation  tool  needed  to 
fulfill  M&S  requirements  was  a 
Portable  Hardware  in  the  Loop 
(PHITL)  facility  which  has  the  capa¬ 
bility  to  test  CMWS  performance 
during  stressing  aircraft  maneuvers 
including  snap  rolls.  The  PHITL  con¬ 
tains  3  CMWS  sensors  and  an  ECU 
mounted  on  the  inner  three  axes  of  a 
five  axis  flight  motion  simulator.  The 
inner  three  axes  simulate  aircraft 
roll,  pitch,  and  heading  while  the 
outer  two  axes  simulate  missile  angu¬ 
lar  motion  (azimuth  and  elevation) 
relative  to  the  aircraft.  A  MEON 
missile  simulator  (hand  held  version 
of  SMEOS)  was  mounted  on  an  opti¬ 


cal  bench  on  the  outer  two  axes  to 
produce  UV  missile  irradiance  on  the 
sensors.  A  collimating  mirror  was 
used  to  eliminate  parallax. 

The  SIIRCM  system  successfully  com¬ 
pleted  all  test  requirements  in  CY 
2001-CY  2002  including  the  follow¬ 
ing  open  air  range  tests:  Developmental 
Test  Potential  False  Alarm  Source  Test 
(DTPFAS),  Captive  Seeker  Test,  Sled 
Test,  Cable  Target  Missile  Firing 
(CTMF-1),  and  Dynamic  Target 
Missile  Firing  Test  (DTMF).  The  pur¬ 
pose  of  the  DTPFAS  test  was  to  meas¬ 
ure  system  performance  in  the  presence 
of  PFAS  sources  and  to  measure  PFAS 
UV  and  IR  source  signatures  for  inclu¬ 
sion  in  the  E2E  model.  PFAS  sources 
tested  and  characterized  included  a 
wide  variety  of  battlefield  and  non-bat¬ 
tle  field  sources  of  UV  and  IR  radia¬ 
tion.  The  purpose  of  the  captive  seeker 
test  was  to  test  the  ability  of  an  air¬ 
borne  installed  SIIRCM  system  to 
defeat  ground  deployed  missile  seeker 
heads  with  the  ATIRCM  Jam  Head 
and  AIIRCM  Flares.  The  purpose  of 
the  sled  test  was  to  demonstrate  that  an 
airborne  installed  SIIRCM  system 
could  track  and  put  jam  energy  on  a 
missile  moving  down  a  sled  track.  The 
purpose  of  the  CTMF-1  test  was  to 
demonstrate  that  a  SIIRCM  system 
mounted  on  an  unmanned  cable  car  at 
the  WSMR  Aerial  Cable  Range  could 
defeat  live  missiles.  The  purpose  of  the 
DTMF  test  was  to  demonstrate  that 


the  CMWS  missile  warning  receiver, 
mounted  on  an  unmanned  drone  QF-4 
aircraft,  could  detect  and  declare  live 
missiles  in  time  for  them  to  be  defeated 
by  dispensed  flares. 

A  comparison  of  the  four  simulation 
tools  to  CTMF-1  test  results  is 
shown  in  Figure  3. 

During  the  CTMF-1  test,  1 1  missiles 
were  fired  at  the  SIIRCM  system 
while  installed  on  a  pod  approxi¬ 
mately  700  feet  in  the  air.  The  pod 
was  attached  to  a  cable  trolley  which 
allowed  the  pod  to  be  moving  for 
some  missile  firings  and  static  for  oth¬ 
ers.  An  IR  heat  source  hung  below 
the  pod  as  a  target  for  the  missiles. 
The  trial  numbers  of  the  11  missile 
firings  are  shown  along  the  bottom 
axis  of  Figure  3.  The  vertical  axis  is 
the  warning  time  (time  until  missile 
impact)  when  the  CMWS  system 
declared  the  incoming  missile  and  ini¬ 
tiated  a  countermeasure  (jammer 
and/or  flare).  The  warning  time  has 
been  normalized  to  avoid  classifica¬ 
tion  problems.  The  four  simulation 
tools  compare  very  well  to  the  actual 
test  data.  Input  irradiances  to  the 
SMEOS  and  PHITL  were  obtained 
from  a  UV  radiometer  on  board  the 
cable  car.  Radiometer  data  was  not 
obtained  for  trials  1  and  lib. 
SMEOS  simulations  were  accom¬ 
plished  on  site  at  the  CTMF-1  test 


but  only  attempted  for  trials  5  and  6. 
PHITL  simulations  were  each  repeat¬ 
ed  24  times  to  assess  the  variability  in 
system  performance  due  to  naturally 
occurring  photon  (Poisson)  noise. 
Input  irradiances  for  the  SIL  and  E2E 
Model  were  predicted  by  the  E2E. 
The  problem  which  caused  the  E2E 
Model  not  to  declare  on  trials  5  and 
6  is  currently  being  investigated. 

In  summary,  the  simulation  tools 
have  proved  successful  in  meeting 
program  goals  for  evaluation  of  the 
SIIRCM  system  under  varying  condi¬ 
tions.  The  primary  difficulties  lie  in 
keeping  simulation  tools  updated  to 
reflect  system  hardware  and  software 
changes.  No  one  tool  was  able  to 
meet  all  of  the  SIIRCM  program’s 
M&S  requirements.  Correlation  is 
critical  for  the  success  of  validation 
and  accreditation  efforts.  Acquisition 
programs  utilizing  simulation  tools 
require  discipline,  control,  and  for¬ 
mal  configuration  management.  ♦ 

Mr.  James  Hatfield  III  is  the  Chief,  IRCM 
Technical  Branch  &  Chief  Systems 
Engineer  for  the  U.S.  Army's  IRCM 
Program.  He  is  responsible  for  planning, 
programming,  scheduling,  budgeting,  and 
direction  for  overall  technical  issues  for 
electronics  equipment  onto  rotary  and 
fixed  wing  aircraft.  M r.  Hatfield  has  over  1 5 
years  experience  within  the  U.S.  Army  pro¬ 
viding  test  and  simulation  support  and  has 


been  providing  support  to  the  SIIRCM  pro¬ 
gram  since  1998  as  lead  for  all 
Developmental  and  Operational  test 
events  and  more  recently,  as  Chief  Systems 
Engineer.  He  may  be  reached  at 
jimhatfield@  peoavn.redstone.army.mil. 

Dr.  Mike  Neer  is  President  of  Neer,  Inc. 
located  in  Belle  Meade,  New  Jersey.  He  is 
the  original  author  Of  the  Organics  Under 
Simulated  Interstellar  Conditions 
(OSIC)  UV  Atmospheric  Propagation 
Model,  a  primary  author  of  the  Tactical 
Missile  Signatures  Measurements 
Standard  Handbook,  and  has  over  25 
years  of  experience  in  IRCM  modeling  and 
testing.  He  is  currently  providing  test  and 
signature  measurement  support  to  the 
SIIRCM  program.  He  received  his  Ph.D.  in 
Aeronautical  and  Astronautical  Engineering 
from  the  Ohio  State  University.  He  may  be 
reached  at  mike@  neerinc.com 

Ms.  Julie  Locker  is  Division  Chief  for 
Software  Engineering  and  Test  at  Morgan 
Research  Corporation  in  Huntsville, 
Alabama.  Ms.  Locker  is  currently  the  con¬ 
tractor  test  and  engineering  lead  for  PM 
Aviation  Electronic  Systems  supporting  the 
U.S.  Army's  SIIRCM  program.  Ms.  Locker 
has  a  Bachelor's  degree  in  Mechanical 
Engineering  from  Georgia  Institute  of 
Technology.  She  may  be  reached  at 
JLocker@  morganres.com. 


Figure  3  CTMF-1  Test  Results 
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Credibility  Assessment  of 

MANPADS  Hit-Point  Predictions 

■  by  M r.  Greg  Czarnecki,  and  Mr  Al  Boyd  (Defense  Intelligence  Agency/Missile  &  Space  Intelligence  Center) 


Aircraft  vulnerability 

reduction  features  are 
typically  distributed 
throughout  the  vehicle 
based  on  expectations  of  random  hits 
from  Anti-Aircraft  Artillery  (AAA) 
and  missile  warhead  fragments.  Until 
recently,  direct  surface-to-air  missile 
(SAM)  hits  were  considered  kills  and 
received  little  vulnerability  attention. 
Results  from  Desert  Storm  changed 
this  mindset.  While  several  classes  of 
aircraft  hit  by  shoulder-launched  man 
portable  air  defense  system(s)  (MAN- 
PADS)  survived  with  frequency,  others 
did  not.  The  deciding  factor  involved  a 
combination  of  missile  hit-points  and 
their  proximity  to  critical  aircraft  sub¬ 
systems.  Current  MANPADS  model¬ 
ing  and  simulation  (M&S)  efforts  have 
been  designed  for  the  purpose  of 
advancing  susceptibility  reduction 
measures.  These  simulations  common¬ 
ly  terminate  with  a  report  of  infrared 
(IR)  countermeasure  effectiveness.  The 
capability  of  MANPADS  M&S  to 
accurately  predict  hit-point  locations 
for  vulnerability  assessment  requires 
further  investigation. 

Acknowledging  the  importance  of 
hit-point  prediction,  the  Joint 
Technical  Coordinating  Group  on 
Aircraft  Survivability  (JTCG/AS) 
launched  a  MANPADS  Impact  Point 
Assessment  project  to  assess  the  abil¬ 
ity  of  current  MANPADS  M&S  to 
accurately  predict  hit-points  on  com¬ 
plex  IR  targets.  This  effort  is  led  by 
Greg  Czarnecki  (46th  Test  Wing)  and 
Al  Boyd  (Defense  Intelligence 
Agency/Missile  &  Space  Intelligence 
Center,  [DIA/MSIC]).  The  project 
uses  a  model-test-model  approach. 
Organizations  providing  M&S  sup¬ 
port  consist  of  MSIC  (Redstone 
Arsenal,  Alabama);  the  Army 
Research  Laboratory-White  Sands 
Missile  Range  (ARL-WSMR);  the 


Threat  Systems  Management  Office 
(TSMO,  Redstone  Arsenal, 
Alabama);  the  Naval  Air  Warfare 
Center  (NAWC  T-SPIL,  China  Lake, 
California);  the  46th  Test  Wing 
(46TW  GWEF,  Eglin  AFB,  Florida); 
the  Air  Force  Information  Warfare 
Center  (AFIWC,  Fackland  AFB, 


Texas);  the  Air  Force  Research 
Faboratory  (AFRF/SN,  WPAFB, 
Ohio);  and  the  Air  Force  Electronic 
Warfare  Evaluation  Simulator 
(AFEWES,  Fort  Worth,  Texas).  Each 
participating  organization  brings  a 
unique  and  diverse  set  of  M&S  solu¬ 
tion  methodologies  to  the  table. 


These  methodologies  range  from  dig¬ 
ital  solutions  to  various  hardware-in- 
the-loop  (HITL)  simulations. 

The  MANPADS  Impact  Point 
Assessment  project  consists  of  three 
phases — 

•  Baseline  Assessment  (FY01) 

•  Collaborative  M&S  Study 
(FY02) 

•  Final  Exam  (FY03) 

Phase  I  used  selected  M&S  hit-point 
prediction  methods  to  design  and 
subsequently  predict  the  results  of  a 
live  fire  test  using  simple  target 
boards.  Shown  in  Figure  1  (see  page 
16),  IR  target  boards,  consisting  of 
three  horizontally-aligned  subtargets 
with  1  meter  spacings,  were  fabricat¬ 
ed  for  use  as  targets.  Figure  2  shows  a 
MANPADS  launch  at  a  target  board. 
M&S  hit-point  predictions  were  com¬ 
pared  with  the  actual  tests  results. 

In  Phase  II,  the  M&S  team  is  partici¬ 
pating  in  a  collaborative  M&S  study. 
This  M&S  study  will  examine  a  spe¬ 
cific  set  of  C- 1 3  0/M  ANPAD  S 
engagement  scenarios.  The  Phase  II 
study  will  culminate  in  documenta¬ 
tion  of  the  MANPADS  M&S  capa¬ 
bilities  for  providing  hit-point  loca¬ 
tions.  Factors  determining  hit-point 
include  the  M&S  ability  to  represent 
various  target  signatures,  the  diversi¬ 
ty  of  MANPADS  used,  and  complex¬ 
ity  of  using  simulations  to  predict 
hit-points  as  opposed  to  only  assess¬ 
ing  IR  countermeasures.  Because 


each  M&S  solution  imploys  different 
methods  for  generating  a  aircraft’s 
signature  and  for  representing  MAN- 
PADS,  the  project  will  use  a  well- 
defined  series  of  M ANPAD  S-air craft 
engagement  scenarios  to  compare  the 
various  M&S  methods. 

In  Phase  III,  the  M&S  team  will 
design  a  series  of  complex  IR  target 
boards  for  final  M&S  evaluation. 
The  target  boards  will  be  fabricated 
and  subjected  to  a  new  series  of 
MANPADS  live  fire  tests.  Test  results 
will  be  compared  to  M&S  hit-point 
predictions.  Given  successful  demon¬ 
stration  of  endgame  solution  credi¬ 
bility,  the  vulnerability  community 
will  be  able  to  use  the  capability  of 
MANPADS  fly-out  M&S  to— 

1.  Significantly  decrease  the  num¬ 
ber  of  views/shotlines  necessary 
for  M  ANPAD  S-air  craft  vulner¬ 
ability  assessments. 

2.  Select  meaningful  MANPADS- 
aircraft  hit  points  for  live  fire 
testing. 

3.  Point  toward  areas  in  need  of 
vulnerability  reduction  attention. 

Hit-point  prediction  capability  will 
be  established  by  the  JTCG/AS  as  an 
essential  element  of  aircraft  vulnera¬ 
bility  reduction.  ♦ 


M  r.  Greg  Czarnecki  received  his  B.S.  in  Civil 
Engineering  and  M.S.  in  Materials 

Engineering  from  the  University  of  Dayton. 
He  is  a  civilian  with  the  46th  Test  1/1/ ing's 
Aerospace  Survivability  and  Safety  Flight. 
Mr.  Czarnecki  is  the  Chairman  of  the 
Structures  Committee  under  the  JTCG/AS 
Vulnerability  Reduction  Subgroup.  He  may 
be  reached  at  gregory.czarnecki@ 
wpafb.af.mil. 

Mr.  Al  Boyd  received  his  B.S.  in  Applied 
Physics  from  the  Georgia  Institute  of 

Technology  and  a  M.S.  in  Engineering 
Management  from  the  University  of 

Alabama ,  Huntsville.  He  is  a  civilian  with 
the  Defense  Intelligence  Agency  Missile 
and  Space  Intelligence  Center,  Office  for 
Defensive  Weapons,  where  he  leads  a 
team  that  develops  EO/IR  SAM  simula¬ 
tions.  He  may  be  reached  at 

wab@  msic.dia.mil. 
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1  2002  Threat  Warheads  and 

1  Effects  Seminar 

C 

■  by  Major  Mark  Carteaux  and  Lt  Col  Anthony  Brindisi 


The  shriek  of  Man  Portable 
Air  Defense  Systems 
(MANPADS)  and  the 
thudding  of  large  caliber 
machine  guns  told  the  story — the 
fourth  Threat  Warheads  and  Effects 
Seminar  had  come  to  town.  This  year, 
from  29  April  to  3  May  2002,  the 
Joint  Services  Air  Defense  Lethality 
Team  (JSADLT)  brought  their  Joint 
Technical  Coordinating  Group  for 
Aircraft  Survivability  (JTCG/AS) 
sponsored  Threat  Warheads  and 
Effects  Seminar  to  the  Hurlburt  Base 
Theater  at  Hurlburt  AFB,  Florida. 
Time  was  split  between  Hurlburt 
AFB  and  Eglin  AFB  for  the  various 
briefings,  live  fire  demonstrations, 
and  capabilities  tours.  As  always,  the 
seminar’s  principal  objective  is  to 
improve  the  warfighter’s  understand¬ 
ing  of  threat  warheads  and  their 
effects  to  U.S.  combat  operations. 

The  seminar  is  structured  around  mul¬ 
timedia  training,  integrating  threat 
lethality  information,  imagery,  hard¬ 
ware  from  threat  exploitation  and  live 
fire  testing,  and  actual  combat  engage¬ 
ments.  It  covers  the  threat  weapons 
systems  from  small  arms  through  the 
latest  Russian  surface-to-air  missile  sys¬ 
tems.  The  technical  portion  of  the  sem¬ 
inar  is  a  collaborative  effort  between 
the  Joint  Services  Air  Defense  Lethality 
Team,  Defense  Intelligence  Agencies, 
46th  Test  Wing,  Air  Force  Special 
Operations  School,  and  Federal 
Aviation  Administration  (FAA).  The 
Missile  and  Space  Intelligence  Center 
(MSIC)  provided  numerous  outstand¬ 
ing  technical  threat  briefings  by  senior 
analysts  and  an  eye  popping  MAN- 
PADS  hardware  display.  The  National 
Ground  Intelligence  Center  (NGIC) 
provided  an  attention-grabbing  brief¬ 
ing  on  Air  Defense  Artillery  to  include 
the  Skyshield  Air  Defense  System, 
which  is  designed  to  counter  both  air¬ 


craft  and  a  wide  range  of  air  delivered 
munitions.  This  year,  the  seminar 
added  the  Air  Force  Research 
Laboratory  (AFRL)  Human 
Effectiveness  Directorate  who  led  a  ses¬ 
sion  on  the  low  power  laser  threat;  and 
responding  to  the  Army’s  experiences 
in  the  war  on  terrorism,  an  introduc¬ 
tion  to  Rocket  Propelled  Grenades 
(RPGs).  Air  Force  Special  Operations 
School  conducted  the  Small  Arms  live 
fire  demonstrations  at  their  Hurlburt 
AFB  range,  and  at  Eglin  AFB  the 


MANPADS  live  fire  demonstrations 
were  held  at  its  B-75  Range  complex. 

As  shown  in  Figure  1,  this  Stinger  live 
fire  shot  was  at  a  C-130  Wing  section 
approximately  5000  feet  downrange. 
Note  the  missile  homing  in  on  the  tar¬ 
get — a  direct  hit.  After  the  seminar  con¬ 
cluded,  many  attendees  were  treated  to 
highly  informative  Chicken  Little  threat 
systems’  capabilities  tour,  a  46th  Test 
Wing’s  capabilities  overview,  the 
Guided  Weapon’s  Evaluation  Facility 
tour,  the  McKinley  Climactic  Hangar 


Figure  I.  MANPADS  Launch 


Figure  2.  MANPADS  Homecoming  in  on  C-130  Wing  Box. 


tour,  and  the  Test  Site  A-22  (Guns) 
tour. 

The  two  live  fire  demonstrations  pro¬ 
vided  a  unique  experience  to  the 
attendees.  The  demonstrations  were 
split  over  two  days,  with  the  small 
arms  demo  on  Tuesday,  30  April 
2002,  and  the  MANPADS  and  RPG 
demonstrations  held  on  Wednesday, 
1  May  2002.  Air  Force  Special 
Operations  sharpshooters  demon¬ 
strated  the  effectiveness  of  small 
arms  (from  5.56  mm  rifles  to  14.5 
mm  heavy  machine  guns)  on  numer¬ 
ous  aircraft  components  and  person¬ 
nel  protective  equipment.  This  par¬ 
ticular  demonstration  always 
changes  how  conference  attendees 
view  the  small  arms  threat.  The 
SHORAD  office  (Stinger  Systems 
Program  Office)  demonstrated  the 
MANPADS  threat.  A  total  of  six 
Stingers  were  fired  against  targets 
ranging  from  intense  infrared  signa¬ 
ture  sources  on  a  plywood  back¬ 
ground  to  a  portion  of  a  C-130  wing 
and  two  Kiowa  helicopters. 
Unfortunately,  one  Stinger  malfunc¬ 
tioned  causing  this  portion  of  the  live 
fire  demonstration  to  be  concluded 
before  the  RPGs  could  be  fired. 

146  personnel  attended  this  year’s 
seminar  from  all  three  Services  and 
industry.  Specifically  represented  were 
Air  Combat  Command,  Air 
Education  Training  Command,  Air 
Mobility  Command,  Air  Force  Special 
Operations  Command,  the  Intelli¬ 
gence  Schoolhouse,  the  Office  of 
Naval  Intelligence,  China  Lake  Naval 
Air  Warfare  Center,  U.S.  Army 
Evaluation  Center,  and  the  Army’s 
Special  Operations  Group.  A  wide 
cross  section  of  the  Military  was  rep¬ 
resented  with  personnel  from  the 
headquarters  level  through  opera¬ 
tional  combat  squadrons  including 
flight  crews,  intelligence  officers, 
Aircraft  Battle  Damage  Repair 
Specialists,  fighters,  bombers,  trans¬ 
porters,  tankers,  and  rescue  groups 
from  Active  Duty,  Guard,  and 
Reserve. 

The  common  thread  throughout  the 
seminar  is  the  emphasis  on  what  type 
of  damage  to  U.S.  combat  aircraft 
occurs  when  one  is  hit  by  threat 
weapons  systems  and  how,  by  careful 
documentation  and  analysis  of  all 


aspects  of  successful  combat  engage¬ 
ments,  we  can  learn  to  more  com¬ 
pletely  protect  our  combat  aircrews 
and  make  our  aircraft  less  vulnerable 
to  enemy  fire.  This  data  can  also 
serve  to  reduce  the  vulnerability  of 
future  weapons  systems.  All  new 
Military  aircraft  acquisition  pro¬ 
grams  have  design-to-requirements 


that  reduce  the  susceptibility  and  vul¬ 
nerability  of  these  future  aircraft. 
Many  more  airmen  have  been  able  to 
return  to  friendly  territory  due  to  the 
emphasis  on  collecting  the  threat 
induced  aircraft  damage  or  lost  air¬ 
craft  data,  then  redirecting  the  intel¬ 
ligence  information  back  to  designers 
and  manufacturers,  as  well  as  war 


Figure  3.  SMSgt  Walter  Briefing  Small  Arms  Live  Fire  Demo 


Figure  4.  Stinger  Shot  Hitting  First  Target 


a> 

E 

E 


-Q 

5 


a 


24 


planners,  operators,  maintainers,  and 
intelligence  personnel.  This  would 
not  have  been  possible  without  the 
teams  of  highly  trained  and  experi¬ 
enced  military  and  civilian  personnel 
who  support  and  recognize  the  need 
for  combat  data  collection  and 
assessment  efforts. 

The  JTCG/AS,  a  long  time  supporter 
and  user  of  combat  data,  recognized 
the  team’s  value  to  all  the  Services  and 
sponsors  many  of  the  Team’s  efforts. 
Under  the  JTCG/AS  mentorship, 
bridges  were  built  between  all  three 
Services  in  order  to  develop  a  tri- 
Service  combat  data  collection  capa¬ 
bility.  Currently,  the  team  is  made  up 
of  reservists  from  Air  Force  Material 
Command’s  46th  Test  Wing 
Aerospace  Survivability  Flight  at 
Wright  Patterson  AFB,  Ohio;  Naval 
Air  Systems  Command,  Patuxent 
River,  Maryland;  Office  of  Naval 
Intelligence/SPEAR,  Washington 
D.C.;  Naval  Air  Warfare  Center, 
China  Lake,  California;  and  the 
Army’s  Aberdeen  Test  Center.  The 
teaming  between  the  components  is 
growing  and  a  true  “purple”  capabili¬ 
ty  is  emerging,  providing  a  strong  and 
diverse  combat  data  collection  force. 

Today,  the  Joint  Service  Air  Defenses 
Lethality  Team  continues  its  vital 
mission.  This  program  provides 
expertise  on  the  lethality  of  threat  air 
defense  systems  against  U.S.  Army, 
U.S.  Navy,  and  U.S.  Air  Force  air¬ 
craft  by  combining  information  from 
threat  exploitation,  live  fire  test,  and 
combat  data  collection.  The  mission 
is  to  enhance  the  warfighter  pre¬ 
paredness  for  threat  damage  to  air¬ 
craft.  This  team  accomplishes  its  mis¬ 
sion  through  three  separate,  yet 
interrelated  projects — 

•  “Battle  Damage  Data 
Collection”  of  combat  data 

•  “MANPADS  Live  Fire 
Evaluation”  guidance  and  par¬ 
ticipation  in  survivability  S&T 
and  live  fire  testing 

•  “Threat  Warheads  and  Effects” 
peacetime  training  to  the 
warfighter  (intelligence,  tactics, 
and  logistics) 


The  first  two  projects  are  certainly 
important  to  further  our  analysis 
capabilities,  understand  the  lethality 
of  threat  projectiles  and  missiles,  and 
compile  complete  records  for  detailed 
analyses,  all  of  which  populates  the 
Survivability/V ulnerability  Information 
Analysis  Center  (SURVIAC)  database. 
However,  it  is  through  the  archives 
contained  in  this  database  and  the 
knowledge  gained  through  these  first 
two  projects  that  the  Threat 
Warheads  and  Effects  training  pro¬ 
gram  has  evolved.  The  primary  train¬ 
ing  objectives  for  the  JSADLT  has 
been  to  provide  the  warfighter  an 
understanding  of  the  lethality  of 
threat  projectiles  and  missiles.  There 
are  a  number  of  classes  of  threats  due 
to  concentration  of  the  projectiles  and 
the  severity  of  the  damage.  The  team 
provides  an  eye  opening,  threat  effects 
experience  of  the  various  kinds  of 
damage  (penetration,  explosion,  and 
collateral)  that  could  truly  ruin  a 
warfighters  whole  day  even  from 
“small  arms”  fire.  Other  learning 
objectives  are  to  train  combat  data 
collectors  for  effective  acquisition  of 
intelligence  data  from  an  incident, 
best  practices  for  obtaining  the  obser¬ 
vations  and  accounts  from  the  pilot’s 
perspective  and/or  their  wingman’s 
perspective,  and  efficient  collection 
procedures  of  the  vehicle  damage  (to 
include  photos  and  the  HUD  video), 
the  maintenance  reports  generated 
(AFTO  97),  and  if  possible  interviews 
with  the  crew  chief  and  other  mainte¬ 
nance  personnel.  Still,  the  biggest 
challenge  is  how  to  best  integrate  any 
and  all  information  gathered  to  accu¬ 
rately  reconstruct  the  incident.  Lastly, 
the  team  must  convey  the  criticallity 
of  recognizing  the  “signature”  of  the 
various  threat  warheads  on  the  dam¬ 
aged  vehicles.  This  truly  is  not  a  sim¬ 
ple  task  and  should  not  be  taken  light¬ 
ly.  A  combat  data  collector  must 
know  where  to  look  for  damage  based 
on  initial  assessments  of  the  vehicle, 
with  the  intent  to  reconstruct  the 
events  that  lead  up  to  the  event,  while 
still  keeping  in  mind  for  his  safety  and 
the  safety  of  others  that  there  is  a  pos¬ 
sibility  of  the  presence  of  unexploded 
ordinance  (UXO)  still  remaining  in  or 
on  the  vehicle.  The  JSADLT  training 
program  effectively  administers  a  real, 
and  most  often  sobering  portrayal  of 
the  threat  effects  from  small  arms  fire, 
anti-aircraft  artillery  fire  (AAA),  and 


man  portable  air  defense  systems 
(MANPADS).  It  is  very  important  to 
the  team  that  they  maintain  a  very 
active  presence  with  the  warfighter 
community  and  continues  to  interface 
with  them  to  provide  the  latest  and 
greatest  threats  that  may  be  or  could 
be  employed  by  an  adversary.  ♦ 

For  additional  details  on  the  Threat 
Warheads  and  Effects  (TW&E)  Seminars , 
please  contact  Maj  Mark  Carteaux, 
USAFR,  at  mark.carteaux@  wpafb.af.mil. 
As  a  reservist  for  the  46th  Test  Wing  at 
W  right  Patterson  AFB ,  Ohio ,  M  aj  Carteaux 
was  the  2002  TW&Es  Seminar  Chairman 
responsible  for  the  planning  and  execution 
of  the  Joint  Technical  Coordinating  Group 
on  Aircraft  Survivability  sponsored  Joint 
Service  Air  Defense  LethalityTeam  premier 
training  event.  Fie  has  a  B.S.  degree  in 
Aeronautical  Engineering  from  Purdue 
University  and  a  M.S.  degree  from  Embry 
Riddle  Aeronautical  University.  Maj 
Carteaux  has  over  11  years  of  U.S.  Air 
Force  active  duty  U.S.  Air  National  Guard, 
U.S.  Air  Force  Reserve,  and  Civilian 
(Lockheed  Martin  and  Government  Ser¬ 
vice)  experience  as  an  Aerospace  Engineer 
and  Aircraft  Maintenance  Officer.  Fie  cur¬ 
rently  works  on  developing  advanced  mate¬ 
rials  and  structural  components  and  on 
improving  Supportability  features  of  both 
RCS  and  IR  signature  reduction  technolo¬ 
gies.  As  a  member  of  the  JSADLT,  Maj 
Carteaux  is  responsible  for  coordinating 
and  integrating  the  efforts  of  the  U.S.  Navy 
into  the  joint  service  team. 

For  additional  details  on  the  training  oppor¬ 
tunities  available  from/with  the  Joint 
Service  Air  Defense  LethalityTeam,  please 
contact  the  team  leader,  Lt  Col  Anthony 
Brindisi,  USAFR,  at  anthony.brindisi@ 
wpafb.af.mil.  Lt  Col  Anthony  Brindisi  is  the 
senior  reservist  for  the  46th  Test  Wing  at 
Wright  Patterson  AFB,  Ohio.  Fie  has  a  B.S. 
degree  from  Parks  College  of  Aeronautical 
Technology  and  holds  an  M.S.  from  the 
University  of  Southern  California.  In  his 
civilian  life,  Tony  has  over  20  years  of  air¬ 
craft  susceptibility  reduction  experience, 
primarily  as  a  Low  Observable  engineer 
working  both  in  industry  and  for  the 
Government.  Fiaving  been  active  in  per¬ 
forming  combat  data  collection  and  analy¬ 
sis  during  Operation  Allied  Force,  he  now 
leads  the  Joint  Service  Air  Defense  Lethality 
Team,  coordinating  efforts  with  the  U.S. 
Army,  U.S.  Navy,  and  U.S. Air  Force  to  per¬ 
form  combat  data  collection. 
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Dry  Bay  Fire  Model  Enhancements 


Meeting  the  Difficulty 


■  by  M r.  Martin  L  Lentz,  Mr  Andrew  Pascal,  and  Mr  Michael  R.Weisenbach 


Fire  onboard  an  aircraft  is  often 
a  deadly  scenario.  Unfor¬ 
tunately,  dry  bays — compart¬ 
ments  or  internal  volumes 
located  between  the  exterior  surface  of 
a  vehicle  and  its  internal  structures — 
may  contain  fluid  lines  (fuel,  hydraulic, 
etc.),  hot  bleed-air  ducts,  electrical  wire 
bundles,  and  intervening  clutter  (struc¬ 
tures,  boxes,  etc.)  that  are  especially 
vulnerable  to  fire.  Examples  of  dry 
bays  common  to  many  aircraft  include 
wing  leading  edge  bays,  landing  gear 
wheel  wells,  avionics  equipment  bays, 
and  engine  accessory  bays. 

In  the  mid-to-late  1980s,  although  a 
great  deal  of  time  and  resources  had 
been  spent  designing  and  evaluating 
tradeoffs  in  fuel  system  protection  rel¬ 
ative  to  risk  reduction,  performance, 
weight,  and  cost,  no  robust  accepted 
model  existed  for  predicting  fires  in 
aircraft.  The  Joint  Technical 
Coordinating  Group  for  Aircraft 
Survivability  (JT CG/AS) — recognizing 
how  all  Military  services  would  bene¬ 
fit  from  a  physics-based,  validated, 
standard  fire  prediction  model — 
became  involved  with  the  Dry  Bay  Fire 
Model  (DBFM)  in  the  early  1990s. 

This  article  focuses  on  the  continuing 
development  of  the  DBFM  to  help 
predict  fires  in  Department  of 
Defense  (DoD)  vehicles.  The  model 
has  evolved  each  year,  growing  in 
physical  capabilities,  while  retaining 
its  original  objective — to  provide  a 
fast  running,  easy-to-use,  and  credi¬ 
ble  accurate  model  applicable  to  a 
wide  range  of  target  threat,  impact, 
and  environmental  conditions. 
Enhancement  and  validation  are 
ongoing  and  under  sponsorship  of 
the  JTCG/AS. 


DBFM’s  Present  Configuration 

The  DBFM  simulates  the  events 
occurring  during  the  penetration  of  a 
single  threat  through  a  vehicle  (fixed- 
wing  aircraft,  helicopter,  or 
wheeled/tracked)  dry  bay  and 
impacting  a  flammable  liquid  filled 
container,  either  a  fuel  tank  or  fuel  or 
hydraulic  line.  The  simulation  cur¬ 
rently  addresses  the  principle  phe¬ 
nomenology  of  contained  fires:  liquid 
spray  ignition,  fire  initiation,  fire 
growth,  and  fire  sustainment. 

The  DBFM  uses  basic  physics,  sup¬ 
plemented  by  empirical  data,  and 
several  simplifying  assumptions  to 
address  the  several  distinct  stages  of 
a  dry  bay  fire.  These  stages  include — 

1.  Initial  Threat  Impact:  Ballistic 
penetration  with  changes  in 
velocity,  mass,  and  direction. 

2.  Ignition  Source:  Armor  pene¬ 
trating  incendiary  (API)  jacket 
striping,  fragment  adiabatic 
shear  “plugging”  and  fragment 
breakup,  High  Explosive 
Incendiary  (HEI)  bursting,  elec¬ 
trical  wire  sparking,  and 
Shaped  Charge  plasma  jetting. 

3.  Fuel  Tank  Impact:  Hole  size 
and  crack  growth  based  on 
shock  pressure  and  impulse  and 
cavity  pressure  and  duration. 
Droplet  sizes  depend  on  fluid 
pressure  and  hole  size. 
Determines  fluid  mass  flow  for 
both  spray  and  steady  state 
leaking.  Fooks  at  droplet  pene¬ 
tration  into  the  bay  and  droplet 
size  based  on  Sauter  Mean 
Diameter  (SMD). 

4.  Fire  Ignition:  Droplet  vaporiza¬ 
tion  in  the  spray  involves  simul¬ 


taneous  heat  transfer  by  con¬ 
duction  and  convection  and 
mass  vapor  transfer  to  the  hot 
gas  cloud  by  convection  and 
diffusion.  Determines  probabil¬ 
ity  of  ignition  for  threat  type. 

5.  Fire  Sustainment:  Applies  con¬ 
servation  equations  for  mass, 
species,  momentum,  and  energy 
to  a  multi-cell,  3 -dimensional, 
dual  grid  system.  Processes 
accomplished  within  each  cell 
are:  forced  convection  from  ven¬ 
tilation  inlet  and  exit  and  dam¬ 
age  hole(s),  buoyancy  in  density 
differences  by  dry  bay  height, 
diffusion,  vaporization  using 
convective  mass  transfer  coeffi¬ 
cient,  liquid  fuel  characteristics, 
flow  velocity  within  dry  bay, 
and  vapor  temperature  above 
surface,  and  radiation  based  on 
average  temperature.  Outputs  of 
this  stage  are  cell  temperature, 
oxygen  content,  and  fuel  vapor 
content  time  histories. 

6.  Fire  Suppression:  No  capability 
in  current  model.  Army  has 
requested  this  enhancement  for 
FY03  and  the  approach  will 
follow  previous  work  in 
droplet  transport,  gaseous 
agent  flame  interaction,  and 
water  mist  quenching  accom¬ 
plished  under  the  Next 
Generation  Halon  Replacement 
Program  (NGP)  funded  by 
OSD/SERDUP. 

7.  Fire  Migration:  Estimates 
steady  state  flow  of  fuel  from 
the  tank  due  to  piezometric 
head  using  Bernoulli’s  equation. 
Fiquid  poring  out  of  a  contain¬ 
er  will  spread  along  the  floor 
and  be  contained  by  structure 
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barriers  and  the  bay  walls. 
Once  the  flow  has  reached  the 
barrier  height,  it  will  be  allowed 
to  flow  over  and  spread  to  next 
barrier.  The  model  supports  10 
bay  barriers. 

8.  Damage  Accumulation:  Current 
model  does  not  address  thermal 
damage  due  to  heat  flux  or 
structural  burn  through.  Heat 
flux  will  a  measurable  variable 
by  the  end  of  2002. 

9.  Vehicle  Response:  Accounts  for 
all  subsystem  and  system  dam¬ 
age  due  to  various  damage  and 
failure  mechanisms.  This 
enhancement  will  be  looked  at 
after  heat  flux  improvements 
have  been  validated. 

The  DBFM  requires  users  to  describe  the — 

•  Dry  bay — Including  the  dimen¬ 
sions;  material;  ventilation  air 
velocity  and  inlet  and  exit  sizes; 
and  the  number,  size,  and  loca¬ 
tion  of  structural  barriers  present. 

•  Fuel  container — Such  as  tank, 
fuel  line,  or  hydraulic  line.  For 
each  container  type,  the  model 
requires  fuel  level  and  volume, 
tank  material,  or  line  diameter 
and  pressure. 

•  Dry  bay  clutter — Can  also  be 
described  to  simulate  structure, 
internal  components,  and  multi¬ 
ple  dry  bays. 

As  Figure  1  shows,  the  amount  of 
detail  required  can  be  extensive 
depending  on  the  complexity  of  the 
user’s  dry  bay.  Dry  bay  configura¬ 
tions  are  not  all  rectangular  parallel- 
pipeds  or  a  single  bay  as  seen  in 
Figure  2. 

The  complexity  of  the  dry  bay  geom¬ 
etry  and  the  need  to  incorporate  and 
use  existing  vehicle  geometry  descrip¬ 
tions  in  the  DBFM  has  been  debated 
for  close  to  10  years.  The  survivabil¬ 
ity/vulnerability  community  uses  two 
distinct  types  of  geometries  to 
describe  aircraft  and  ground  vehicles. 
The  aircraft  industry  uses  a  form  of 
Finite  Element  grid  similar  to  NAS- 
TRAN  called  FASTGEN,  and  the 
Army  uses  for  its  ground  vehicles  a 


solid  geometry  method  entitled, 
BRL-CAD.  Both  methods  have 
advantages  and  challenges,  but  the 
automatic  processing  of  required 
dimension  data  for  the  DBFM  by 
either  method  remains  a  desired,  cur¬ 
rently  unmet  enhancement. 

A  sustained  fire,  commonly  called  a 
diffusion  flame,  is  where  the  reac¬ 
tants  are  initially  separated  and  are 
brought  together  and  burn  in  a  nar¬ 
row  strip  separating  them.  In  dry  bay 
fires,  the  vaporization  of  the  liquid 
pool  supplies  fuel  vapor,  and  air 
found  initially  in  the  dry  bay  is 
replenished  by  ventilation  air  enter¬ 
ing  the  bay.  To  simulate  the  reacting 
flow,  the  DBFM  uses  the  principles  of 
local  conservation  of  mass,  momen¬ 
tum,  species,  and  energy.  Solutions 
are  obtained  on  a  numerical  grid  of 
elementary  control  volumes  that  fill 
the  entire  dry  bay.  This  approach 
ensures  local  balances  in  the  many 
small  control  volumes,  subject  only 
to  the  particular  boundary  condi¬ 
tions.  To  improve  computational 
efficiency  and  reduce  model  run¬ 
time,  the  DBFM  uses  a  multi-cell, 
three-dimensional  dual  grid  system — 
a  diffusion  and  convection  grid. 
Within  the  diffusion  grid,  the  simula¬ 
tion  of  fire  is  performed  by  the  solu¬ 
tion  of  the  conservation  equations  of 
mass,  momentum,  species,  and  ener¬ 
gy.  Because  convective  currents  work 
over  much  larger  distance  scales  than 
the  process  of  diffusion  and  combus¬ 
tion,  the  conservation  of  momentum 
(i.e.,  the  convective  mass  and  energy 
flow)  is  performed  in  the  convective 
grid  and  the  results  superimposed 
into  the  diffusion  grid  by  the  prorat¬ 
ing  of  flow  seen.  In  this  manner,  the 
user  can  achieve  accuracy  and  reduce 
model  run-times  by  performing  the 
conservation  of  momentum  in  a  larg¬ 
er  grid,  reflecting  the  large  distance 
scales  of  convection  and  performing 
the  remaining  calculations  in  a  small¬ 
er  grid,  reflecting  the  smaller  distance 
scales  of  diffusion  and  combustion. 

The  convection  grid  cell  must  be  larg¬ 
er  or  equal  to  the  diffusion  cell  size 
and  the  diffusion  cell  must  be  an  even 
divisor  of  the  convective  cell  size.  As 
an  example,  for  a  6  inch  convective 
cell,  diffusion  cell  sizes  of  6,  3,  2,  or  1 
inch  are  permissible.  Fractional  sizes, 
such  as  0.5,  0.25,  or  0.1  are  also 


Figure  I.  Dry  Bay  Description 
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Figure  3.  Fire  Initiation  Sequence 

allowed.  When  selecting  a  convective 
cell  size,  the  number  of  cells  will 
reflect  the  dry  bay  dimensions.  The 
number  of  cells  is  the  rounded  integer 
of  the  ratio  of  the  dry  bay  dimensions 
and  cell  size.  Thus,  if  the  dry  bay 
length  is  3.75  feet  and  the  convection 
cell  size  is  6  inches,  the  simulation 
will  run  with  8  cells  (i.e.,  the  model 
assumes  the  bay  is  48  inches  long).  If 
more  accurate  simulation  is  required, 
a  3  inch  cell  width  must  be  used.  In 
that  case,  15  cells  will  exactly  fill  the 
dry  bay  dimension. 


Figure  6.  Dry  Bay  at  8  Seconds 

The  motion  of  fluid  within  the  bay  is 
described  by  the  Eulerian  approach 
where  the  composition,  velocity,  and 
state  of  the  fluid  everywhere  in  the 
bay  at  all  instants  is  determined.  To 
derive  the  Eulerian  equations  of 
motion,  a  stationary  coordinate  sys¬ 
tem  is  chosen  and  fluid  is  observed 
entering  and  exiting  a  volume  of 
arbitrary  shape  and  size.  This  control 
volume  is  found  within  the  fluid  and 
is  at  rest  with  respect  to  the  coordi¬ 
nate  system.  In  the  DBFM,  the  vol¬ 
ume  is  a  cube  (cell)  whose  length  is 
user-specified. 

To  achieve  its  fast  run-times,  DBFM 
uses  several  simplifying  assumptions, 
including — 

•  Fluid  flow  is  modeled  using 
Bernoulli’s  equation  that  drives 
flow  by  pressure  differences 


Figure  7.  Dry  Bay  at  10  Seconds 

between  cells.  Streamlines  with 
curvature,  recirculation  zones, 
and  eddies  are  not  explicitly 
modeled.  Having  said  that, 
recirculation  zones  are  seen  to 
occur.  Contributing  to  the  devel¬ 
opment  of  recirculation  zones  is 
the  fact  that  viscous  effects 
along  the  wall  of  the  dry  bay  are 
included  in  the  simulation 

•  Fuel  and  air  are  assumed  per¬ 
fectly  mixed  within  each  cell. 
Turbulence  is  not  simulated. 
Complete  (i.e.,  no  soot  is 
formed)  and  perfect  combustion 
(i.e.,  combustion  products  are 
limited  to  C02  and  H20)  of 
fuel  and  air  is  assumed. 

•  Radiation  is  treated  by  emission 
alone,  utilizing  an  “effective” 
emissivity. 


Sample  Case 

Based  on  the  current  version  of  the 
model  (DBFM  3.2)  a  typical  aircraft 
dry  bay  run  for  a  cluttered  dry  bay  or 
two  separate  bays  connected  by  a 
small  opening  (see  Figure  4). 

The  clutter  in  the  bay  is  shown  as 
greenish  blue.  One  piece  is  seen  on 
the  left  and  represents  a  simple  black 
box  (avionics).  On  the  upper  right, 
clutter  represents  structure  in  a  scal¬ 
loped  pattern.  In  the  center,  more 
clutter  splits  the  dry  bay  into  two 
separate  compartments.  Ventilation 
flow  enters  into  the  left  bay  and 
exists  the  right  bay.  Flow  between 
bays  occurs  through  a  narrow  slit  in 
the  center  clutter.  The  threat  impact 
is  in  the  left  bay  perpendicular  to  the 
paper.  Fuel  is  free  to  flow  over  the 
bottom  center  clutter  from  the  left 
bay  into  the  right  one. 

The  figure  also  shows  the  dry  bay  at 
time  zero.  Ignition  has  occurred  and 
the  model  is  initialized  with  hot  com¬ 
bustion  gases  (1, 500-1, 600K)  in  the 
left  bay.  The  cross  section  indicates 
the  slit  between  the  compartments. 

Figure  5  shows  the  dry  bay  5  seconds 
after  ignition.  Fire  is  burning  in  the 
left  bay  and  hot  gases  are  seen 
migrating  into  the  right  bay.  Figure  6 
shows  the  dry  bay  8  seconds  after 
ignition.  A  sustained  fire  is  burning 
in  the  left  bay,  while  the  temperature 
in  right  bay  is  climbing  due  to  the 
migrating  of  hot  gases  following  the 
airflow  from  the  left  to  right  bay.  In 
the  fuel  vapor  density  plot  (center) 
there  is  some  vaporization  of  the  liq¬ 
uid  pool  starting.  Figure  7  is  the  dry 
bay  at  10  seconds  where  a  fire  has 
been  initiated  in  the  right  bay  from 
vaporized  fuel  igniting.  The  oxygen 
concentration  in  the  left  bay  (top 
plot)  is  continuously  diminishing. 
Figure  8  (see  page  28)  shows  the  dry 
bay  20  seconds  after  ignition.  The 
fire  in  the  right  bay  is  starting  to  self 
extinguish  due  to  a  lack  of  oxygen 
while  the  fire  in  the  left  bay  is  reach¬ 
ing  peak  temperatures.  Fresh  air  only 
enters  in  the  left  bay  through  the  vent 
inlet  and  the  damage  hole  from  the 
threat  penetration.  The  fire  in  the  left 
bay  is  consuming  the  majority  of  the 
new  oxygen  and  very  little  is  migrat¬ 
ing  through  the  slit  into  the  right  bay. 
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Figure  8.  Dry  Bay  at  20  Seconds 


If  the  model  were  to  continue,  the 
plots  would  show  the  right  bay  tem¬ 
perature  continuing  to  cool  down 
while  the  left  bay  would  continue  to 
burn  until  no  fuel  was  left. 

Model  Validation 

Several  efforts  are  directed  at  accom¬ 
plishing  DBFM’s  ongoing  verifica¬ 
tion  and  validation.  The  Joint  Strike 
Fighter  (JSF)  program  office  is  fund¬ 
ing  an  effort  to  verify  the  current 
methodology  in  the  DBFM  and  to 
build  a  suite  of  test  cases  for  future 
model  verification.  A  sensitivity 
study  is  ongoing  with  results  planned 
to  be  published  in  late  2002. 

A  separate  program — funded  by 
the  OSD/DOT&E/LFT&E  office 
and  called  Survivability  and  Safety 
Initiative  for  Aircraft  (SSAI) — was 
to  design,  fabricate,  and  test  a  dry 
bay  fire  simulator,  and  use  the  data 
generated  to  validate  fire  models 
for  DoD.  This  program,  which 
started  in  1997  and  ended  in  2001 
involved  all  three  Military  Services 
and  SANDIA  National  Labs. 
Under  guidance  of  a  group  of  fire 
modelers  from  the  services  and 
SANDIA,  a  dry  bay  test  fixture 
was  designed,  instrumented,  and 
constructed  at  the  Aerospace 
Survivability  and  Safety  Flight  at 
Wright-Patterson  AFB,  Ohio. 
Figure  10  shows  the  fixture  in  the 
airflow  facility.  Over  a  two-year 
period,  96  baseline  and  mixing 
tests  were  performed. 


This  test  data  is  being  used  to  validate 
the  DBFM.  Temperature  data  has 
been  compared  and  a  test  report  is 
under  review.  Heat  flux  data  will  be 
examined  in  late  2002.  The  DBFM  is 
continuously  being  used  to  predict  fire 
probability  in  ongoing  live  fire  tests 
for  all  new  vehicle  development  and 
joint  live  fire  tests  programs  of  exist¬ 
ing  aircraft  in  the  inventory.  All  prob¬ 
lems  and  lack  of  capability  due  to 
physics  or  databases  are  reported  to 
the  DBFM  Configuration  and  Control 
Board  (CCB),  which  meet  twice  each 
year.  The  board  and  model  manager 
decide  where  resources  will  be  applied 
to  enhance  the  model  during  the  next 
coming  year.  The  CCB  is  one  of  the 
responsibilities  of  the  JTCG/AS. 

The  JTCG/AS  is  sponsoring  an  addi¬ 
tional  validation  effort  starting  in 
2002  titled  the  Dry  Bay  Fire  Ignition 
Phase  Validation  and 

Experimentation  program.  It  will 
address  the  possible  different  types  of 
ignitions  within  the  dry  bay  such  as 
convective  vs.  radiation  heating  of 
the  fuel  pool,  hot  gas  cloud  expan¬ 
sion  to  touch  the  pool  or  the  steady 
flow  from  the  damage  hole  in  the 
tank,  or  the  secondary  droplet  igni¬ 
tion  from  a  front  face  splash.  This 
test  simulation  will  strive  to  show 
visualization  and  instrumentation  to 
define  the  method  or  methods  of 
ignition  that  occur  in  a  ballistic 
impacted  dry  bay.  Results  are 
planned  for  release  in  early  2004. 

Conclusions 

The  DBFM  is  a  model  under  continu¬ 
ing  development  for  the  application 
of  fire  prediction  in  DoD  vehicles. 
The  model  is  a  tool  requested,  used, 


and  improved  by  a  coordinated  group 
of  survivability/vulnerability  experts. 
The  model’s  applications  do  not 
require  it  to  be  100%  accurate,  only 
credible  as  defined  by  its  users.  That 
is  the  purpose  and  direction  of  its 
continued  enhancement.  ♦ 
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The  JSF  LFT&E  Program 

1  1 

■  by  Mn  James  Rhoads 


Lockheed  Martin’s  Joint 
Strike  Fighter  (JSF)  is  a  cov¬ 
ered  system  as  defined  by  the 
Live  Fire  Test  (LFT)  Law. 
With  this  requirement,  Lockheed 
Martin  was  required  to  develop  a 
process  that  would  not  only  identify 
the  vulnerabilities  of  the  aircraft,  but 
would  also  provide  a  methodology 
for  defining  the  Live  Fire  Test  and 
Evaluation  (LFT&E)  plan.  Lockheed 
Martin  has  pioneered  a  process  that 
combines  the  vulnerability  design 
process,  with  modeling  and  simula¬ 
tion  (M&S)  to  produce  an  in-depth 
LFT  program  that  covers  many  issues 
and  the  spectrum  of  conventional  and 
non-conventional  threats. 

As  written  in  the  previous  issue  by 
Mr.  Hugh  Griffis  (“Vulnerability 
Design  Discipline,”  Aircraft 
Survivability ,  Spring  2002),  acquisi¬ 


tion  reform  introduced  a  perform¬ 
ance  based  vulnerability  requirement 
onto  the  JSF  program.  This  has 
opened  the  trade  space  for  the 
designer  allowing  Lockheed  Martin 
to  pursue  new  technologies  to  solve 
traditional  vulnerability  problems 
and  to  affordably  support  the  vulner¬ 
ability  requirements  while  maintain¬ 
ing  the  balanced  design  approach  for 
the  overall  air  system.  The  tradition¬ 
al  vulnerable  area  and  time  based  kill 
levels  have  been  replaced  with 
engagement  probability  of  kill  (Pk/e) 
and  mission  oriented  kill  levels.  In 
both  instances,  the  new  terminology 
and  definitions  provided  a  clearer 
understanding  for  senior  manage¬ 
ment  and  easier  design  acceptance. 

The  Lockheed  Martin  process  for 
performing  a  vulnerability  assess¬ 
ment  and  the  development  of  the 


LFT&E  plan  is  outlined  in  Figure  1. 
There  are  two  loops  to  this  process 
but  both  loops  begin  with  the  defini¬ 
tion  of  the  requirements  in  the  bot¬ 
tom  of  the  outer  loop.  Once  the 
requirements  are  defined,  trade  stud¬ 
ies  are  conducted  to  determine  which 
design  features  best  reduce  the  vul¬ 
nerability  of  the  aircraft  and  meet  the 
balanced  design  approach.  These 
trade  studies  require  the  development 
of  a  detailed  geometric  target 
description.  Lockheed  Martin  initiat¬ 
ed  a  process  several  years  ago  that 
extracted  the  geometric  data  from 
CATIA,  ACAD,  and  NASTRAN 
databases  to  form  very  realistic  tar¬ 
get  descriptions  with  minimal  labor 
hours.  A  process  that  once  took  6 
months  was  reduced  to  weeks  and  at 
the  same  time  increased  the  level  of 
fidelity  and  accuracy  of  the  target 
description. 


Design  Data 
Conversion 
Programs 

Target  Model 
Development 
Programs 

JSF  Design  Tools 
•ACAD 
•  CATIA 

Modify  Design 
:  •  Trade  Studies 
•  Establishes 

FASTGEN  4 
+ 

PGEN  4 

COVART  4 


Analysis 
Verification  & 
Modification 

Live 

Uncertainty 

Analysis 

Fire 

LFT  Candidate 
Selection 

Live  Fire  Tests 

Test 

+ 

Loop 

Test 

Predictions 

Planning  & 
Scheduling 

Requirements 


Prob. 
of  Kill 
per 
Hit  or 
Burst 


Delta  URF  Cost  &  Weight 


Figure  I.  Lockheed  Martin  Vulnerability  and  LFT&E  Design  Process. 
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Lockheed  Martin  has  implemented 
several  unique  M&S  techniques  to 
refine  and  enhance  the  vulnerability 
process.  These  include  tracking  the 
weight  of  the  components  within  the 
target  model,  number  of  component 
and  interferences,  and  the  size  of  the 
target  description  file.  Tracking  the 
target  model  weight  and  matching  to 
the  real  weights  carried  by  the  mass 
properties  group  insures  proper 
shielding  throughout  the  aircraft.  An 
additional  M&S  enhancement 
included  performing  an  uncertainty 
analysis  on  probability  of  component 
dysfunction  given  a  hit  (Pcd/h) 
tables.  The  Pcd/h  table  is  the  core  to 
the  current  vulnerability  analysis 
process  used  within  COVART. 
Lockheed  Martin  asked  the  question 
“what  happens  if  our  Pcd/h  tables 
are  wrong?”  A  process  was  devel¬ 
oped  that  essentially  involved  run¬ 
ning  the  vulnerability  assessment 
three  times — 


•  The  current  baseline  Pcd/h  table, 


•  Setting  the  Pcd/h  table  to  all  1 .0  values, 

•  Setting  the  Pcd/h  table  to  all  0.0  values. 

The  results  of  the  sensitivity  analysis 
were  combined  with  additional  LFT 
guidelines  to  prioritize  and  develop 
the  overall  LFT  plan. 
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Some  of  the  guidelines  for  developing 
the  LFT  plan  included  but  were  not 
limited  to  testing  those  things  that 
were  considered  invulnerable  but  had 
little  test  data,  unique  components  or 
materials  that  have  no  test  data 
(Electro  Hydraulic  Actuators  [EHAs] 
and  lift  fan  drive  shaft),  the  cost  of 


testing  is  cheaper  than  M&S,  or  those 
components  that  contributed  more 
than  5%  to  the  aircraft’s  Pk/e.  The 
number  of  tests  and  the  test  articles 
were  assembled  with  the  following 
common  goal:  all  tests  performed  on 
a  full-up  test  article  must  have  an  ear¬ 
lier  test  on  a  component  or  subassem¬ 
bly  article.  This  goal  was  established 
to  prevent  surprises  when  testing  is 
begun  on  the  two  aircraft  that  have 
been  acquired  for  LFT  (Navy  drop 
test  article  and  a  Full-up  Short  Take 
Off  and  Vertical  Landing  [STOVL] 
variant).  In  addition  to  the  tradition¬ 
al  ballistic  threats,  the  LFT  plan  cov¬ 
ers  directed  energy  threats  (low 
power  lasers  and  High  Power 
Microwave  [HPM])  as  well  as  an 
extensive  test  series  for  chemical  and 
biological  threats.  Also  because  the 
JSF  will  be  using  a  27  mm  armor 
piercing  projectile  currently  not  field¬ 
ed  within  the  U.S.  inventory,  lethality 
testing  will  be  performed  to  demon¬ 
strate  the  round  and  gun  system  can 
achieve  the  required  ground  target 
probability  of  kill. 

The  argument  for  using  M&S  versus 
testing  was  addressed  while  writing 
the  extensive  Live  Fire  Master  Test 
Plan.  The  LFT  plan  contains  numer¬ 
ous  tests  to  develop  data  for  M&S 
activities  because  there  is  a  lack  of 
confidence  in  current  M&S  activities 
or  there  are  no  models  to  address  the 
specific  vulnerability.  For  example, 
fire  is  the  number  one  contributor  to 
aircraft  vulnerability,  yet  we  have 
very  simplistic  fire  prediction  capa¬ 
bilities.  The  MANPADS  threat  does 
not  have  a  defined  modeling  capabil¬ 
ity.  Several  tests  have  been  performed 
on  full-up  legacy  aircraft  that  have 


shown  common  damage  modes. 
Other  vulnerability  considerations 
such  as  spall  from  the  aircraft  canopy 
or  fuel  migration  are  lacking  in  mod¬ 
eling  capability.  Improvements  in 
vulnerability  engineering  modeling 
and  vulnerability  system  level  model¬ 
ing  are  possible  and  will  be  pursued. 
These  are  a  sample  of  the  M&S 
issues  and  these  have  been  provided 
to  the  JSF  M&S  office  along  with  the 
appropriate  government  M&S 
groups  in  order  to  improve  the  vul¬ 
nerability  M&S. 

In  summary,  Lockheed  Martin  has 
developed  a  comprehensive  LFT&E 
program  for  the  JSF  aircraft.  The  pro¬ 
gram  benefits  from  an  extensive  use  of 
CAD  developed  data  for  target 
descriptions,  refinements  of  the  vul¬ 
nerability  analysis  process,  scrutiny  of 
the  vulnerability  M&S  codes,  and  the 
desire  to  reduce  risk  for  full  up  aircraft 
tests.  Testing  has  already  begun  and 
will  continue  throughout  the  System 
Development  and  Demonstration 
phase  of  the  program.  ♦ 

Mr.  James  Rhoads  received  his  B.S.  in 
Aerospace  Engineering  at  Pennsylvania 
State  University  in  1990.  He  has  been  per¬ 
forming  vulnerability  assessments  on  air¬ 
craft  since  1990  and  joined  Lockheed 
M  artin  in  1997.  Currently  he  is  responsible 
for  writing  the  JSF  LFT  M  aster  Plan ,  leading 
several  of  the  test  programs  as  well  as  con¬ 
ducting  analysis  and  trade  studies.  He  has 
served  on  the  American  Institute  of 
Aeronautics  and  Astronautics  (AIAA) 
Survivability  Technical  Committee  from 
1996-2002.  He  may  be  reached  at 
610.354.2804. 


Educating  Warfighters  and 
Acquisition  Professionals 


On  the  Fundamentals  of  Aircraft  Survivability  Combat  through  VTC 


■  by  CDR  Mark  Couch 


The  current  philosophy 
among  the  warfighters  is 
that  Military  aircraft 
should  not  be  designed  to 
for  a  single  role  or  unique  capability, 
but  rather  all  Military  aircraft  should 
be  used  in  multiple  roles  and  have 
many  capabilities.  As  these  new  air¬ 
craft  are  designed,  acquisition  profes¬ 
sionals  must  consider  more  threats  to 
these  aircraft  in  order  to  ensure  that 
these  new  aircraft  survive  in  combat 
and  complete  their  assigned  missions. 
Additionally,  all  Services  are  seeing 
“mission  creep”  on  all  their  legacy 
aircraft,  mission  creep  defined  as  new 
roles  and  missions  given  to  aircraft 
that  they  were  not  originally  designed 
to  perform.  For  example,  the  U.S.  Air 
Force  is  considering  using  KC-135 
tanker  as  an  airborne  relay  to  update 
target  information  for  fighter  and 
attack  aircraft.  How  will  this  affect 
the  combat  survivability,  the  reliabili¬ 
ty,  or  the  system  safety  of  this  air¬ 
craft?  Will  this  aircraft  become  a  high 
priority  target?  Will  the  aircraft  be 
considered  non-mission  capable  if  the 
airborne  link  system  is  inoperable? 
Normal  procedures  for  airborne  refu¬ 
eling  are  to  minimize  electromagnetic 
transmissions  to  enhance  system  safe¬ 
ty,  but  now  transmissions  will  be 
required  using  some  part  of  the  elec¬ 
tromagnetic  spectrum. 

So  what  do  mission  creep  and  multi¬ 
ple  roles  mean  to  the  acquisition  pro¬ 
fessional  in  regards  to  aircraft  combat 
survivability?  It  means  that  focusing 
on  a  few  small  aspects  of  survivabili¬ 
ty,  such  as  signature  reduction  or  mis¬ 
sile  launch  detection,  are  not  enough 
to  ensure  our  Military  aircraft  survive 
in  combat.  The  warfighters  and 
acquisition  professionals  must  con¬ 
sider  all  aspects  of  aircraft  combat 
survivability  to  include  both  vulnera¬ 
bility  and  susceptibility  reduction 


concepts.  The  question  is  how  do  the 
program  offices  accomplish  this 
seemingly  daunting  task?  Part  of  the 
answer  is  in  the  education  of  both  the 
warfighters  and  the  acquisition  pro¬ 
fessionals.  The  Naval  Postgraduate 
School  offers  the  only  course 
[Aircraft  Combat  Survivability, 
Reliability  and  System  Safety 
Engineering  (AA  4251)]  in  the  coun¬ 
try  that  takes  a  comprehensive  look 
at  aircraft  combat  survivability, 
including  both  vulnerability  and  sus¬ 
ceptibility  reduction  concepts.  Under 
the  sponsorship  of  the  JTCG/AS,  this 
course  is  now  available  via  Virtual 
Teacher  Center  (VTC)  to  personnel  at 
other  Service/government  facilities. 

This  course  is  designed  to  accommo¬ 
date  both  warfighters  and  acquisition 
professionals  and  focuses  on  teaching 
the  fundamentals  of  aircraft  combat 
survivability  without  over-emphasiz¬ 
ing  any  one  particular  attribute  by 
using  The  Fundamentals  of  Aircraft 
Survivability  Analysis  and  Design , 
2nd  Edition,  by  Dr.  Robert  E.  Ball. 
The  course  begins  defining  the  terms 
used  in  aircraft  combat  survivability. 
What  does  survivability  mean? 
Vulnerability?  Susceptibility?  Others? 
Then,  the  students  are  introduced  to 
the  inherent  probabilistic  nature  of 
aircraft  combat  survivability  along 
with  the  requisite  knowledge  of  prob¬ 
ability  theory  needed  to  understand 
the  problems  associated  with  aircraft 
survivability.  The  course  discusses 
measures  of  mission  effectiveness  and 
how  survivability  is  enhanced  through 
the  six  susceptibility  reduction  con¬ 
cepts  (threat  warning,  noise  jamming 
and  deceiving,  signature  reduction 
[stealth],  expendables,  threat  suppres¬ 
sion,  and  tactics)  and  six  vulnerability 
reduction  concepts  (component 
redundancy  with  separation,  compo¬ 
nent  location,  passive  damage  sup¬ 


pression,  active  damage  suppression, 
component  shielding,  and  component 
elimination  or  replacement). 

Next,  students  are  introduced  to  the 
roles  and  missions  of  aircraft  and  the 
air  defense  threats.  Threat  character¬ 
istics  are  identified  and  the  damage 
processes  and  mechanisms  are  dis¬ 
cussed.  Since  most  threat  weapons 
use  some  portion  of  the  electromag¬ 
netic  spectrum,  students  are  intro¬ 
duced  to  the  physics  of  radar  and 
infrared  radiation  to  give  them  a 
basic  understanding  of  the  phenome¬ 
na.  Some  specific  threat  weapons 
from  various  countries  (U.S.,  allies 
and  potential  adversaries)  are  intro¬ 
duced  to  give  the  student  an  appreci¬ 
ation  for  the  numbers,  types,  and 
modes  of  operation  of  air  defense 
weapons  that  could  be  brought  to 
bear  against  an  aircraft,  either  fixed- 
wing  or  rotary-wing.  One  student 
who  took  the  course  in  the  winter  of 
2002  said, 

...the  text  and  course  were  some  of 
the  best  unclassified  sources  for  air¬ 
craft  roles  and  missions  and  air 
defense  threats 

that  she  had  seen. 

Finally,  the  course  addresses  more 
detailed  quantification  of  aircraft 
combat  survivability  to  the  analysis 
and  design  of  Military  aircraft.  In 
this  portion,  the  student  learns  how 
to  conduct  fault  tree  analyses  and 
damage  mode  effects  analyses  in 
order  to  determine  the  critical  com¬ 
ponents  on  an  aircraft.  With  this 
methodology,  the  vulnerability  of  an 
aircraft  can  be  quantified  so  that 
trade  studies  can  be  made  on  various 
vulnerability  reduction  features. 
Similarly,  quantification  of  the  sus¬ 
ceptibility  of  an  aircraft  is  made  to 
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look  at  various  susceptibility  reduc¬ 
tion  concepts.  Finally,  the  vulnerabil¬ 
ity  and  susceptibility  portions  are 
combined  to  quantify  the  survivabili¬ 
ty  of  an  aircraft  against  a  specific 
threat  in  the  given  encounter  condi¬ 
tions. 

In  order  to  give  the  students  an 
appreciation  for  breadth  of  the  air¬ 
craft  combat  survivability  discipline 
and  how  it  influences  other  parts  of 
the  design  process,  segues  are  made 
into  the  reliability  and  system  safety 
disciplines.  Students  are  shown  how 
these  three  disciplines  complement 
each  other  in  the  design  process  and 
methodologies  learned  in  one  disci¬ 
pline  can  easily  be  applied  to  anoth¬ 
er.  It  is  this  part  of  the  course  that 
bridges  the  operational  requirements 
of  the  warfighter  with  the  mainte¬ 
nance  requirements  of  the  acquisition 
professional.  The  goals  of  the  aircraft 
combat  survivability  discipline  are — 


newly  hired  engineers  at  Patuxent 
River  and  other  U.S.  Navy  facilities 
were  expressed,  and  it  was  decided 
that  the  use  Virtual  Teacher  Center 
(VTC)  technology  would  be  the  best 
way  to  reach  these  groups.  It  was 
decided  that  JTCG/AS  would  spon¬ 
sor  the  VTC  portion  of  the  course  for 
winter  quarter  2002  and  to  make  the 
course  available  to  any  interested 
DoD  activity.  Three  sites  decided  to 
participate:  two  from  NAVAIR 
(Patuxent  River  and  China  Lake)  and 
the  JSF  Program  office  in  Crystal 
City.  A  total  of  31  students  took  the 
course  including  the  5  in-resident 
students  from  NPS.  The  VTC  por¬ 
tion  of  the  course  was  taught  concur¬ 
rently  with  the  in-resident  portion  to 
minimize  costs  and  maximize  inter¬ 
action  between  NPS  students  and 
civilian  engineers. 

Some  of  the  comments  from  the  stu¬ 
dents  were — 


l.The  early  identification  and  suc¬ 
cessful  incorporation  of  those 
specific  survivability  enhance¬ 
ment  features  that  increase  the 
combat  cost  effectiveness  of  the 
aircraft  as  a  weapon  system. 


Thanks  to  NPS  (CDR  Couch)  and 
JTCG/AS  for  giving  students  at  distant 
sites  the  opportunity  to  participate  in  the 
class  and  learn  about  such  an  important 
part  of  current  and  future  naval  weapon 
planning  and  development 


2.  In  those  circumstances  where 
the  damage  will  eventually  lead 
to  and  aircraft  kill,  the  surviv¬ 
ability  enhancement  features 
should  allow  a  graceful  degrada¬ 
tion  of  the  system  capabilities, 
giving  the  crew  a  chance  to 
depart  the  aircraft  over  friendly 
territory. 
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Course  AA  4251  was  originally 
developed  for  in-resident  Naval 
Postgraduate  Students  (NPS)  stu¬ 
dents  to  provide  them  with  knowl¬ 
edge  of  aircraft  combat  survivability 
and  how  it  relates  to  system  safety 
engineering  and  reliability  engineer¬ 
ing.  The  concept  of  delivering  the 
course  on  a  continual  basis  to  non¬ 
resident  students  came  from  a  meet¬ 
ing  in  November  2001  with  RADM 
Heely,  (Air  4.0),  Dr.  Bob  Ball,  Mr. 
Ken  Goff  (Head,  Survivability 
Engineering  Division,  Air  4.1.8), 
LCDR  Andy  Cibula  (JTCG/AS),  and 
CDR  Mark  Couch  (course  instruc¬ 
tor)  to  discuss  the  future  of  the  NPS 
survivability  program.  In  this  meet¬ 
ing,  concerns  about  educating  young, 


Excellent  course!  I  was  given  a  good 
introduction  intoACS.Thank  you. 

A  very  worthwhile  course. 

Due  to  the  overwhelming  success  of 
the  first  course,  a  decision  was  made  to 
offer  the  course  again  for  the  Summer 
quarter  2002.  Electronic  announce¬ 
ments  were  sent  to  all  DoD  activities 
involved  with  aircraft  survivability 
(JTCG/AS  mailing  list).  Seven  sites 
decided  to  participate:  three  from 
NAVAIR  (Pax  River,  China  Lake,  and 
Lakehurst),  two  from  Aberdeen 
Proving  Grounds,  and  one  each  from 
Quantico  (supporting  MCOTEA  and 
MCCDC)  and  Wright  Patterson  AFB 
(supporting  46th  OG,  AFMC,  and 
ASC).  The  course  is  again  being  taught 
concurrently  with  the  in-resident  stu¬ 
dents  with  a  total  of  91  students  (12  at 
NPS).  To  demonstrate  how  survivabil¬ 
ity  can  be  applied  to  systems  engineer¬ 
ing  studies,  the  NPS  students  will  be 
conducting  survivability  analyses  for 
Expeditionary  Warfare  2020 
CONOPS  project  currently  underway 
as  part  of  the  systems  engineering  inte¬ 


gration  (SEI)  curriculum  at  NPS.  CDR 
Bill  Erhardt  (SEI  Team  Leader)  is  tak¬ 
ing  AA  4251  and  agreed  to  serve  as 
team  leader  for  the  survivability  proj¬ 
ect.  The  purpose  of  the  survivability 
project  is  to  show  how  conducting  sur¬ 
vivability  analyses  early  in  the  design 
process  can  improve  the  overall  design 
of  the  air  vehicle.  ♦ 

CDR  Mark  Couch  received  a  B.S.  in 
Chemical  Engineering  and  a  B.S.  in 
Chemistry  in  1984  from  Purdue  University. 
After  graduation ,  he  reported  to  Aviation 
Officer  Candidate  School  in  Pensacola , 
Florida  and  was  commissioned  as  an 
Ensign,  and  immediately  reported  to  NAS 
Whiting  Field  to  commence  primary  flight 
training.  Upon  completion  of  helicopter 
flight  training,  he  reported  to  Helicopter 
Mine  Countermeasures  Squadron  14 
(HM-14)  and  served  in  the  Operations 
and  Maintenance  Departments.  During 
this  tour,  the  squadron  was  rapidly 
deployed  to  the  Arabian  Gulf  to  conduct 
mine  countermeasures  operations  in  sup¬ 
port  of  Operation  Earnest  Will.  After  com¬ 
pletion  of  this  tour,  he  reported  to  U.S. 
Atlantic  Fleet  to  serve  as  the  Aide  and  Flag 
Lieutenant  to  the  Deputy  and  Chief  of 
Staff.  In  1 991,  he  reported  to  NPS  to  begin 
graduate  studies  in  Aeronautical  Engineer¬ 
ing.  He  received  a  MS.  in  Aeronautical 
Engineering  (with  distinction)  in  September 
1993.  His  research  was  in  the  field  of 
rotary-wing  unsteady  aerodynamics,  and  he 
co-authored  two  papers.  After  completing 
his  studies,  he  reported  to  HM-14  again 
and  served  as  the  Aviation  Safety  Officer 
during  a  Class  A  mishap  investigation.  He 
later  reported  to  the  Helicopter  Tactical 
Wing,  U.S.  Atlantic  Fleet  where  he  served 
as  the  W ing  Operations  Officer  and  Safety 
Officer.  In  1997,  he  reported  back  to  HM- 
14  for  a  third  tour  as  the  Operations 
Officer  and  Administrative  Officer.  In  1999, 
he  reported  to  NPS  where  he  serves  in  his 
present  assignment  as  military  faculty  in 
the  Department  of  Aeronautics  and 
Astronautics.  He  currently  teaches  gradu¬ 
ate-level  courses  in  Aircraft  Combat 
Survivability,  Computational  M  ethods,  and 
Aircraft  Fatigue.  Additionally,  he  is  pursuing 
his  doctorate  in  Aeronautical  Engineering 
concurrent  with  his  military  duties.  CDR 
Couch  has  approximately  1500  flight 
hours  in  the  T-34C,  TH-57B/C,  RH-53D, 
and  MH-53E  aircraft  with  300  hours 
under  tow.  He  has  received  four  Navy 
Commendation  Medals,  a  Navy 
Achievement  Medal  and  four  Battle  "E" 
Awards. 


Aircraft  Combat  Survivability,  Reliability,  and 
Systems  Safety  Engineering  (AA  4251) 


DoD  Engineers  and  Analysts:  Are  you  interested  in  learning  more  about: 

Aircraft  Combat  Survivability  |  Reliability  Engineering  |  Systems  Safety  Engineering 


The  Naval  Postgraduate  School  (NPS),  under  the  sponsor¬ 
ship  of  the  JTCG/AS,  offers  a  distance  learning  course  in 
Aircraft  Combat  Survivability,  Reliability  and  Systems 
Safety  Engineering  (AA  4251).  This  course  is  offered  via  Virtual 
Teacher  Center  (VTC)  and  gives  the  practicing  engineer  or  analyst 
the  tools  necessary  to  solve  problems  in  these  disciplines. 

Course  Description 

This  course  provides  an  understanding  of  the  essential  elements  in 
the  study  of  survivability  reliability  and  systems  safety  engineering  for 
military  aircraft  (fixed  wing,  rotary  wing,  and  unmanned  air  vehicles). 
Also  presents  technologies  for  increasing  survivability  and  method¬ 


ologies  for  assessing  the  probability  of  survival  in  a  hostile  (non¬ 
nuclear)  environment  of  anti-aircraft  artillery  (AAA)  and  surface-to- 
air  missiles  (SAM).  Several  in-depth  studies  of  the  survivability  of 
fixed  wing  and  notary  wing  aircraft  will  give  the  student  practical 
knowledge  in  the  design  of  battle-ready  military  aircraft.  An  introduc¬ 
tion  to  reliability  and  system  safety  engineering  examines  aircraft  sys¬ 
tem  and  subsystem  failure  in  a  non-hostile  environment.  Safety  analy¬ 
ses  (hazard  analysis,  fault-tree  analysis,  and  component  redundancy 
design),  safety  criteria,  and  lifecycle  considerations  are  presented  with 
applications  to  aircraft  maintenance,  repair;  and  retirement  strategies 
along  with  the  mathematical  foundations  of  statistical  sampling,  set 
theory,  probability  modeling,  and  probability  distribution  functions. 


Who? 

Aircraft  survivability  engineers,  aircraft  design  engineers,  modeling  and  simulation  analysts,  or  anyone 
interested  in  aircraft  survivability  testing  or  design.  Both  new  and  experienced  personnel  can  benefit 
from  this  course  as  it  covers  all  aspects  of  aircraft  survivability  design  from  susceptibility  reduction,  vul¬ 
nerability  reduction,  and  modeling  and  simulation  analysis.  Students  of  the  course  usually  include  a  mix 
of  military  officers  with  flight  experience  and  aircraft  engineers  from  all  Services. 

When: 

Twice  yearly,  usually  in  the  Summer  (July-September)  and  Winter  (January-March)  quarters.  Class 
meets  twice  weekly  for  2  hours.  Specific  days  and  times  are  coordinated  with  curriculum  requirements 
at  the  NPS.  Special  one-week  course  offerings  are  available  upon  request  based  upon  the  availability  of 
the  course  instructor  and  the  number  of  interested  students. 

Where: 

In  aVTC  room  near  you!  Exact  locationsTBD  by  demand.  Sites  previously  served  include:  Paxtuxent 
River;  China  Lake,  Wright  Patterson  AFB,  Edwards  AFB,  Aberdeen  Proving  Ground,  and  several  loca¬ 
tions  in  the  Washington,  DC  area. 

Registration: 

The  Summer  session  for  2002  is  closed  to  new  applicants,  lif  there  is  interest  in  future  course  partici¬ 
pation,  please  fill  out  the  course  registration  form  and  fax  it  to  either  the  JTCG/AS  Central  Office 
703.604.1033  or  the  NPS  Instructor  at  the  address  provided  below. 

Fee: 

DoD  employees  (Military  and  civilian)  -  No  cost.  DoD-Contractors  -  $1000. 

This  course  is  sponsored  by  the  Joint  Technical  Coordinating  Group/Aircraft  Survivability  (JTCG/AS). 

Each  VTC  site  will  bear  the  costs  of  connecting  to  the  NPS  VTC  bridge. 

Instructor: 

CDR  Mark  Couch,  NPS,  Phone:  83  1 .656.2944,  Fax:  83  1 .656.7884,  E-mail:  macouch@nps.navy.mil 

Prerequisites: 

B.S.  in  Engineering  or  Science. You’ll  be  taught  any  graduate  level  math  and  physics  that  you’ll  need. 

Graduate  Credit: 

Students  may  take  this  course  for  4  graduate  credit. Those  interested  in  receiving  graduate  credit  are 
required  to  take  two  exams  and  complete  a  project  in  aircraft  combat  survivability. The  course  may  be 
taken  as  a  non-credit  course,  no  exams  or  projects  are  required. 

Registration  Form 

Name: _ Organization: _ 

Position/Code:  _ Telephone:  _ 

E-mail: _ Type  Aircraft  Involved  With  or  Flown: 

Do  you  want  to  take  the  course  for  □  Credit  □  Non-credit  □  Don’t  know  yet 

Other  Information  you  want  me  to  know:  _ 
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■  by  M n  Kevin  Crosth waite 


The  JTCG/AS  is  pleased  to  recognize  Mr.  Jeffrey 
Wuich,  an  associate  at  Booz  Allen  Hamilton 
who  supports  the  Survivability /Vulner ability 
Information  Analysis  Center  (SURVIAC),  as 
our  second  Young  Engineer  in  Survivability.  Jeff  is  one  of 
SURVIAC’s  bright  young  engineers  who  is  doing  an  out¬ 
standing  job  supporting  the  Joint  Live  Fire  (JLF),  Live  Fire 
Test  &  Evaluation  (LFT&E),  and  JTCG/AS  programs  as 
well  as  answering  day-to-day  survivability/vulnerability/ 
lethality  technical  questions  as  a  SURVIAC  representative. 


Jeff  is  the  SURVIAC  lead  engineer  responsible  for  sup¬ 
porting  vulnerability  and  lethality  testing  conducted  on 
U.S.  Army,  U.S.  Navy,  U.S.  Air  Force,  and  U.S.  Marine 
Corps  aircraft.  He  has  over  10  years  of  professional  expe¬ 
rience  evaluating  new  and  existing  aircraft  to  combat 
threats  in  support  of  a  number  of  Joint  Live  Fire  (JLF)  and 
Live  Fire  Test  and  Evaluation  (LFT&E)  Programs.  Jeff  has 
authored  numerous  detailed  test  plans  and  final  reports, 
aircraft  battle  damage  and  repair  (ABDR)  reports,  and  has 
assisted  in  developing  LFT&E  strategies,  waiver  packages, 
and  alternative  test  plans  required  by  congressional  law. 
Aircraft  (fixed-wing  and  rotary)  he  is  familiar  with  include 
the  A-10,  AH-1S,  AH-64,  AV-8B,  F-14,  F-15,  F-16, 
F/A-18,  F-22,  B-l,  C-5,  C-17,  C-130,  Comanche, 
CH-47,  CH-53,  Mi-24,  MiG-23,  MiG-29,  and  UH-60. 
Aircraft  systems  he  is  familiar  with  include  avionics,  crew 
station,  flight  controls,  fuel  systems,  hydraulics,  propul¬ 
sion,  and  structures.  Combat  threats  he  is  familiar  with 
include  armor  piercing  incendiary  (API)  projectiles,  high 
explosive  incendiary  (HEI)  projectiles,  man-portable  air 
defense  systems  (MANPADS),  vehicle  launched  surface- 
to-air  missiles  (SAMs),  and  air-intercept  missiles  (AIMs). 
Damage  mechanisms  he  is  familiar  with  include  projectile 
penetration  and  blast,  spall,  hydrodynamic  ram,  inlet 
ingestion,  dry  bay  fire,  and  ullage  explosion.  Jeff  provides 
outstanding  administrative  and  technical  support  to  the 
programs  he  supports. 

Prior  to  working  for  Booz  Allen  in  support  of  SURVIAC, 
Jeff  served  as  an  officer  in  the  U.S.  Air  Force.  Jeff  served 
as  an  aerospace  engineer  and  was  assigned  to  the  Wright 
Laboratories,  Flight  Dynamics  Lab,  Vehicle  Subsystems 
Division,  Survivability  Enhancement  Branch  (now  the  46 
OG/OGM/OL-AC).  Jeff  was  responsible  for  evaluating 
new  aircraft  configurations  and  materials  to  determine 
their  vulnerability  to  combat  threats.  He  also  conceived, 


planned,  and  conducted  ballistic  tests  to  discover  and 
reduce  aircraft  vulnerability.  Jeff  served  as  the  Program 
Manager  for  the  F-16  portion  of  the  $3.5  million  Joint 
Live  Fire  (JLF)  Test  Program  where  he  was  responsible  for 
all  vulnerability  testing  conducted  on  the  USAF’s  F-16 
Fighting  Falcon.  Jeff  was  also  a  member  of  the  Combat 
Operations  Assessment  and  Reporting  Program  (CO ARP) 
Team  that  was  trained  to  collect  data  on  U.S.  aircraft  that 
received  combat  damage  during  Operation  Desert  Storm. 
Although  the  COARP  Team  never  made  it  in  country,  they 
did  travel  to  each  of  the  units  to  collect  combat  data  (e.g., 
interview  pilots  and  ABDR  technicians  and  take  photo¬ 
graphs  of  damaged  and  repaired  aircraft)  upon  their 
return  from  overseas.  The  U.S.  Air  Force  has  served  him 
well  as  it  helps  him  to  fully  understand  each  of  the  Services 
he  works  with  on  a  daily  basis  as  well  as  he  is  able  to 
apply  all  that  he  learned  from  his  Military  career  to  his 
current  career. 

Currently,  Jeff  is  the  Data  Manager  for  the  Joint  Live  Fire 
-  Aircraft  Systems  Program.  He  provides  administrative 
and  technical  support  to  the  U.S.  Air  Force,  U.S.  Army, 
U.S.  Navy,  Foreign  Aircraft  Deputy  Test  Director’s,  and 
the  JLF/ Air  Joint  Test  Director.  He  assists  in  preparing 
annual  JLF/Air  Test  Program  proposals,  Grey  Book 
entries,  and  publication  of  JLF/Air  detailed  test  plans  and 
final,  reports.  He  manages  current  JLF/Air  efforts  includ¬ 
ing  the  Standard  Characterization  of  Foreign  Ammunition 
(SCOFA)  and  the  JLF/LFT  Information  System.  Jeff 
authored  the  Test  Approach  and  Strategy  for  the  JLF/Air 
U.S.  20  mm  PGU-28/B  SAPHEI  vs.  Foreign  Fixed-Wing 
Aircraft  effort.  This  effort  proved  to  be  “ground-break¬ 
ing”  for  JLF/Air’s  role  in  supporting  the  Joint  Munitions 
Effectiveness  Manuals  (JMEM’s)  for  the  warfighter.  Jeff 
recently  reported  on  the  JLF  Program  in  a  well-written 
article  entitled  “The  JLF  Program”  in  the  Winter 
2001/2002  issue  of  the  JTCG  Aircraft  Survivability 
newsletter  and  the  SURVIAC  Bulletin.  Jeff  is  sometimes 
referred  to  as  the  “glue”  behind  the  JLF/Air  Program  since 
he  provides  most  of  the  grunt  work  required  to  keep  the 
program  organized,  on  schedule,  and  moving  forward. 

Jeff  also  supports  the  46th  Test  Wing  and  the  C-5 
Reliability  Enhancement  and  Re-engineering  (RERP) 
Development  System  Office  (DSO),  both  located  at 
Wright-Patterson  AFB,  Ohio,  with  the  C-5  Modernization 
LFT&E  Program.  Jeff  supported  the  C-5  RERP  DSO  with 


their  LFT&E  waiver  package  and  alternative  test  plan.  He 
is  currently  assisting  the  C-5  DSO  and  the  46 
OG/OGM/OL-AC  (the  responsible  test  organization 
(RTO)  for  the  Air  Force)  with  each  of  the  detailed  test 
plans  required  for  component  and  sub-system  level  test 
series.  Jeff  was  a  key  support  member  for  the  C-17  and 
B-l  CMUP  LFT&E  Programs.  The  C-17  was  the  very 
first  U.S.  Air  Force  aircraft  to  undergo  Five  Fire  Testing. 
His  knowledge  of  completed  LFT&E  programs,  the  issues 
of  concern  and  the  history  behind  them,  the  various  test 
organizations  involved,  the  decision  makers,  and  the  poli¬ 
tics  sometimes  involved  with  LFT&E  make  him  a  valuable 
asset  to  any  program  office. 

Jeff  is  the  SURVIAC  point  of  contact  for  the  FASTGEN, 
COVART,  FATEPEN,  BEAMS,  WINFIRE,  and  UFFEX 
computer  models.  These  models  are  funded  by  the 
JTCG/AS  &:  JTCG/ME.  Jeff  assists  in  planning  and  con¬ 
ducting  the  Fire  &  Explosion  and  Penetration  Models 
Configuration  Control  Board  (CCB)  meetings  and  is 
responsible  for  preparing  and  distributing  the  minutes. 
CCB  meetings  provide  an  open  forum  for  people  from  the 
S/V/F  community  to  present  their  program  needs,  findings 
from  using  the  model,  model  deficiency  reports,  etc.,  in 
order  to  continuously  enhance  the  capabilities  and  improve 
the  accuracy  of  the  models.  Ultimately,  these  meetings  pro¬ 
vide  guidance,  planning,  and  future  direction  for  the  mod¬ 
els.  Jeff  is  responsible  for  documenting  the  progress  made 
and  plays  a  key  role  in  making  sure  the  aforementioned 
models  keep  improving. 

Jeff  received  his  B.S.  in  Aerospace  Engineering  (1988) 
from  Iowa  State  University  and  his  M.S.  in  Mechanical 
Engineering  (1992)  from  the  University  of  Dayton.  He  is  a 
member  of  the  National  Defense  Industrial  Association 
(NDIA)  and  currently  possesses  a  security  clearance.  Jeff  is 
also  a  professional  engineer  (P.E.)  registered  in  the  state  of 
Ohio  (1995)  and  a  certified  energy  manager  (C.E.M.- 
1996)  as  recognized  by  the  Association  of  Energy 
Engineers.  He  has  over  five  years  of  energy  management 
experience  and  has  conducted  over  three-hundred  energy 
surveys  for  various  utility,  commercial,  and  industrial 
clients.  Jeff  has  been  married  for  eight  years  to  his  fun  and 
loving  wife  Julie.  He  has  two  children,  Claire  (age  5)  and 
Luke  (age  3)  who  provide  him  with  hours  of  fun,  love,  and 
some  of  life’s  more  interesting  challenges.  His  hobbies  and 
interests  include  boating,  water-skiing,  and  snow-skiing. 
His  stress  relievers  include  running,  weight-lifting,  swim¬ 
ming,  and  hot-tubbing.  It  is  with  great  pleasure  that  we 
present  Mr.  Jeffrey  Wuich  as  the  second  person  to  receive 
the  JTCG/AS  Young  Engineer  in  Survivability  recognition. 
♦ 

Mr.  Kevin  Crosthwaite  is  Director  of  the  Survivability/Vulnerability 
Information  Analysis  Center  (SURVIAC).  He  has  worked  on  several 
technical  analysis  and  test  programs  involving  a  wide  variety  of 
weapon  systems.  M  r.  Crosthtwaite  has  an  M  .S.  in  nuclear  physics 
from  Ohio  State  and  is  a  licensed  professional  engineer.  He  serves 
on  the  ADPA  Combat  Survivability  Executive  board  on  the  AIAA 
Survivability  Technical  Committee.  He  may  be  reached  at 
937.255.4840,  DSN  785.480,  or  via  E-mail  at  crosthwaite_ 
kevin@  bah.com. 


Aircraft 

Survivability 

Symposium 

18-21  November  2002 


Sponsored  by — 

•  The  National  Defense 
Industrial  Association  (NDIA) 
and  the  Association  for 
Unmanned  Vehicle  Systems 
International 

•  Naval  Postgraduate  School, 
Monterey,  California 

Goals  of  Symposium — 

•  To  acquaint  aircraft  survivability 
professionals  with  the  characteristics  of 
current  and  planned  UAV  systems  as  well 
as  operational  concepts 

•  To  bring  the  UAV  community  up-to-date 
on  the  survivability  techniques  and 
technologies  embodied  in  manned  aircraft 
for  hit  avoidance  and  damage  mitigation. 


For  more  information,  visit 
www.ndia.org  or  call 
703.522.1820. 
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Calendar  of  Events 


SEPT 

8-13,  Toronto,  Canada 

23rd  International  Congress  of  the 
Aeronautical  Sciences 

vwvw.AAIA.org 

16-18,  Huntsville,  AL 

5th  Conference  on  Aerospace 
Materials,  Processes,  and 
Environmental  Technology 
(AM  PET) 

am  pet.  m sfc .  n asa.gov 


0(T 

1-2,  Panama  City,  FL 

40th  Annual  Air  Targets,  UAVs,  and 
Range  Operations 

www.ndia.org 

29-1  Nov,  Woburn,  MA 

Aircraft  Fires  and  Explosions  due 
to  Accidents,  Combat,  and 
Terrorist  Attacks 

www.blazetech.com/ 

Cou  rse_Listi  ngs/cou  rsej  isti  ngs.htm  I 


4-8,  Dayton,  OH 

Aircraft  Fire  Protection/Mishap 
Investigation  Course 

members.aol.com/afp  I  fire/www.htm 

1 8-2 1 ,  Monterey,  CA 

Aircraft  Survivability  2002 — 
Combat  Survivability:  UAVs  and 
Manned  Aircraft 

www.ndia.org 


Information  for  inclusion  in  the 
Calendar  of  Events  may  be  sent  to: 

SURVIAC,  Washington  Satellite  Office 
Attn:  Christina  McNemar 
3190  Fairview  Park  Drive,  9th  Floor 
Falls  Church,  VA  22042 
PHONE:  703.289.5464 

FAX:  703.289.5467 


COMMANDER 

NAVAL  AIR  SYSTEMS  COMMAND  (4.1.8  J) 

47123  BUSE  ROAD 

PATUXENT  RIVER,  MO  20670-1547 


Official  Business 
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